Release Notes - HBase - Version 1.5.0 - HTML format

Sub-task

  • [HBASE-17884] - Backport HBASE-16217 to branch-1
  • [HBASE-19631] - Allow building HBase 1.5.x against Hadoop 3.0.0
  • [HBASE-19932] - TestSecureIPC in branch-1 fails with NoSuchMethodError against hadoop 3
  • [HBASE-20048] - Revert serial replication feature
  • [HBASE-20716] - Unsafe access cleanup
  • [HBASE-21616] - Port HBASE-21034 (Add new throttle type: read/write capacity unit) to branch-1
  • [HBASE-21675] - Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
  • [HBASE-21679] - Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
  • [HBASE-21680] - Port HBASE-20194 (Basic Replication WebUI - Master) and HBASE-20193 (Basic Replication Web UI - Regionserver) to branch-1
  • [HBASE-21727] - Simplify documentation around client timeout
  • [HBASE-21735] - Port HBASE-18784 (Use of filesystem that requires hflush / hsync / append / etc should query outputstream capabilities) to branch-1
  • [HBASE-21838] - Create a special ReplicationEndpoint just for verifying the WAL entries are fine
  • [HBASE-22132] - Backport HBASE-22115 "HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe" intent to branch-1
  • [HBASE-22413] - Backport 'HBASE-22399 Change default hadoop-two.version to 2.8.x and remove the 2.7.x hadoop checks' to branch-1
  • [HBASE-22458] - TestClassFinder fails when run on JDK11
  • [HBASE-22506] - Backport HBASE-22500 'Modify pom and jenkins jobs for hadoop versions' to branch-1
  • [HBASE-22535] - TestShellRSGroups fails when run on JDK11
  • [HBASE-22554] - Upgrade to surefire 2.22.2
  • [HBASE-22627] - Port HBASE-22617 (Recovered WAL directories not getting cleaned up) to branch-1
  • [HBASE-22706] - Backport HBASE-21292 "IdLock.getLockEntry() may hang if interrupted" to branch-1
  • [HBASE-22719] - Add debug support for github PR pre commit job
  • [HBASE-22728] - Upgrade jackson dependencies in branch-1
  • [HBASE-22891] - Use HBaseQA in HBase-PreCommit-GitHub-PR job
  • [HBASE-22988] - Backport HBASE-11062 "hbtop" to branch-1
  • [HBASE-23101] - Backport HBASE-22380 to branch-1
  • [HBASE-23110] - Backport HBASE-23054 "Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm" to branch-1

Bug

  • [HBASE-16499] - slow replication for small HBase clusters
  • [HBASE-18116] - Replication source in-memory accounting should not include bulk transfer hfiles
  • [HBASE-18312] - Ineffective handling of FileNotFoundException in FileLink$FileLinkInputStream.tryOpen()
  • [HBASE-18517] - limit max log message width in log4j
  • [HBASE-19240] - Fix error-prone errors, part four?
  • [HBASE-19850] - The number of Offline Regions is wrong after restoring a snapshot
  • [HBASE-20006] - TestRestoreSnapshotFromClientWithRegionReplicas is flakey
  • [HBASE-20691] - Storage policy should allow deferring to HDFS
  • [HBASE-20744] - Address FindBugs warnings in branch-1
  • [HBASE-20764] - build broken when latest commit is gpg signed
  • [HBASE-20870] - Wrong HBase root dir in ITBLL's Search Tool
  • [HBASE-20928] - Rewrite calculation of midpoint in binarySearch functions to prevent overflow
  • [HBASE-21021] - Result returned by Append operation should be ordered
  • [HBASE-21070] - SnapshotFileCache won't update for snapshots stored in S3
  • [HBASE-21132] - return wrong result in rest multiget
  • [HBASE-21208] - Bytes#toShort doesn't work without unsafe
  • [HBASE-21575] - memstore above high watermark message is logged too much
  • [HBASE-21740] - NPE happens while shutdown the RS
  • [HBASE-21749] - RS UI may throw NPE and make rs-status page inaccessible with multiwal and replication
  • [HBASE-21777] - "Tune compaction throughput" debug messages even when nothing has changed
  • [HBASE-21796] - RecoverableZooKeeper indefinitely retries a client stuck in AUTH_FAILED
  • [HBASE-21866] - Do not move the table to null rsgroup when creating an existing table
  • [HBASE-22047] - LeaseException in Scan should be retired
  • [HBASE-22057] - Impose upper-bound on size of ZK ops sent in a single multi()
  • [HBASE-22145] - windows hbase-env causes hbase cli/etc to ignore HBASE_OPTS
  • [HBASE-22185] - RAMQueueEntry#writeToCache should freeBlock if any exception encountered instead of the IOException catch block
  • [HBASE-22194] - Snapshot unittests fail on Windows due to invalid file path uri
  • [HBASE-22230] - REST Server drops connection on long scans
  • [HBASE-22378] - HBase Canary fails with TableNotFoundException when table deleted during Canary run
  • [HBASE-22396] - Backport HBASE-21536 to branch-1
  • [HBASE-22426] - Disable region split/merge switch doen't work when 'hbase.assignment.usezk' is set true
  • [HBASE-22492] - HBase server doesn't preserve SASL sequence number on the network
  • [HBASE-22495] - Update SyncTable section, explaining from which specific minor versions "doDeletes/doPuts" is available
  • [HBASE-22510] - Address findbugs/spotbugs complaints (branch-1)
  • [HBASE-22519] - New Hadoop 2.8 dependencies fail shaded invariants check
  • [HBASE-22520] - Avoid possible NPE in HalfStoreFileReader seekBefore()
  • [HBASE-22538] - Prevent graceful_stop.sh from shutting down RS too early before finishing unloading regions
  • [HBASE-22559] - [RPC] set guard against CALL_QUEUE_HANDLER_FACTOR_CONF_KEY
  • [HBASE-22562] - PressureAwareThroughputController#skipControl never invoked
  • [HBASE-22563] - Reduce retained jobs for Jenkins pipelines
  • [HBASE-22571] - Javadoc Warnings related to @return tag
  • [HBASE-22586] - Javadoc Warnings related to @param tag
  • [HBASE-22601] - Misconfigured addition of peers leads to cluster shutdown.
  • [HBASE-22603] - Javadoc Warnings related to @link tag
  • [HBASE-22605] - Ref guide includes dev guidance only applicable to EOM versions
  • [HBASE-22629] - Remove TestReplicationDroppedTables from branch-1
  • [HBASE-22637] - fix flaky TestMetaTableMetrics test
  • [HBASE-22649] - Encode StoreFile path URLs in the UI to handle scenarios where CF contains special characters (like # etc.)
  • [HBASE-22653] - Do not run errorProne on JDK7
  • [HBASE-22654] - apache-rat complains on branch-1
  • [HBASE-22656] - [Metrics] Tabe metrics 'BatchPut' and 'BatchDelete' are never updated
  • [HBASE-22658] - region_mover.rb should choose same rsgroup servers as target servers
  • [HBASE-22666] - Add missing @Test annotation to TestQuotaThrottle
  • [HBASE-22686] - ZkSplitLogWorkerCoordination doesn't allow a regionserver to pick up all of the split work it is capable of
  • [HBASE-22715] - All scan requests should be handled by scan handler threads in RWQueueRpcExecutor
  • [HBASE-22720] - Incorrect link for hbase.unittests
  • [HBASE-22730] - XML Parsing error on branch-1
  • [HBASE-22735] - list_regions may throw an error if a region is RIT
  • [HBASE-22773] - when set blockSize option in Performance Evaluation tool, error occurs:ERROR: Unrecognized option/command: --blockSize=131072
  • [HBASE-22774] - [WAL] RegionGroupingStrategy loses its function after split
  • [HBASE-22784] - OldWALs not cleared in a replication slave cluster (cyclic replication bw 2 clusters)
  • [HBASE-22801] - Maven build issue on Github PRs
  • [HBASE-22838] - assembly:single failure: user id or group id 'xxxxx' is too big
  • [HBASE-22856] - HBASE-Find-Flaky-Tests fails with pip error
  • [HBASE-22861] - [WAL] Merged region should get its WAL according to WALProvider.
  • [HBASE-22866] - Multiple slf4j-log4j provider versions included in binary package (branch-1)
  • [HBASE-22902] - At regionserver start there's a request to roll the WAL
  • [HBASE-22935] - TaskMonitor warns MonitoredRPCHandler task may be stuck when it recently started
  • [HBASE-22937] - The RawBytesComparator in branch-1 have wrong comparison order
  • [HBASE-22955] - Branches-1 precommit and nightly yetus jobs are using jdk8 for jdk7 jobs
  • [HBASE-22981] - Remove unused flags for Yetus
  • [HBASE-23007] - UnsatisfiedLinkError when using hbase-shaded packages under linux
  • [HBASE-23015] - Replace Jackson with relocated gson everywhere but hbase-rest
  • [HBASE-23019] - Handle --skip-errorprone on branch-1
  • [HBASE-23086] - TestShell failing on branch-1 and branch-1.4
  • [HBASE-23094] - Wrong log message in simpleRegionNormaliser while checking if merge is enabled.
  • [HBASE-23128] - Restore Region interface compatibility
  • [HBASE-23139] - MapReduce jobs lauched from convenience distribution are nonfunctional

New Feature

Improvement

  • [HBASE-17314] - Limit total buffered size for all replication sources
  • [HBASE-19024] - Configurable default durability for synchronous WAL
  • [HBASE-19036] - Add action in Chaos Monkey to restart Active Namenode
  • [HBASE-19290] - Reduce zk request when doing split log
  • [HBASE-19673] - Backport " Periodically ensure records are not buffered too long by BufferedMutator" to branch-1
  • [HBASE-20087] - Periodically attempt redeploy of regions in FAILED_OPEN state
  • [HBASE-20109] - Add Admin#getMaster API to branch-1
  • [HBASE-20209] - Do Not Use Both Map containsKey and get Methods in Replication Sink
  • [HBASE-20286] - Improving shell command compaction_state
  • [HBASE-20406] - HBase Thrift HTTP - Shouldn't handle TRACE/OPTIONS methods
  • [HBASE-20444] - Improve version comparison logic for HBase specific version string and add unit tests
  • [HBASE-20446] - Allow building HBase 1.x against Hadoop 3.1.x
  • [HBASE-20450] - Provide metrics for number of total active, priority and replication rpc handlers
  • [HBASE-20845] - Support set the consistency for Gets and Scans in thrift2
  • [HBASE-21000] - Default limits for PressureAwareCompactionThroughputController are too low
  • [HBASE-21126] - Add ability for HBase Canary to ignore a configurable number of ZooKeeper down nodes
  • [HBASE-21164] - reportForDuty to spew less log if master is initializing
  • [HBASE-21256] - Improve IntegrationTestBigLinkedList for testing huge data
  • [HBASE-21283] - Add new shell command 'rit' for listing regions in transition
  • [HBASE-21318] - Make RefreshHFilesClient runnable
  • [HBASE-21325] - Force to terminate regionserver when abort hang in somewhere
  • [HBASE-21328] - add HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP switch to hbase-env.sh
  • [HBASE-21595] - Print thread's information and stack traces when RS is aborting forcibly
  • [HBASE-21659] - Avoid to load duplicate coprocessors in system config and table descriptor
  • [HBASE-21810] - bulkload support set hfile compression on client
  • [HBASE-21883] - Enhancements to Major Compaction tool
  • [HBASE-21932] - Use Runtime.getRuntime().halt to terminate regionserver when abort timeout
  • [HBASE-21987] - Simplify RSGroupInfoManagerImpl#flushConfig() for offline mode
  • [HBASE-21996] - Set locale for javadoc
  • [HBASE-22009] - Improve RSGroupInfoManagerImpl#getDefaultServers()
  • [HBASE-22034] - Backport HBASE-21401 and HBASE-22032 to branch-1
  • [HBASE-22283] - Print row and table information when failed to get region location
  • [HBASE-22301] - Consider rolling the WAL if the HDFS write pipeline is slow
  • [HBASE-22344] - Document deprecated public APIs
  • [HBASE-22363] - Remove hardcoded number of read cache block buckets
  • [HBASE-22377] - Provide API to check the existence of a namespace which does not require ADMIN permissions
  • [HBASE-22459] - Expose store reader reference count
  • [HBASE-22464] - Improvements to hbase-vote script
  • [HBASE-22593] - Add local Jenv file to gitignore
  • [HBASE-22596] - [Chore] Separate the execution period between CompactionChecker and PeriodicMemStoreFlusher
  • [HBASE-22604] - fix the link in the docs to "Understanding HBase and BigTable" by Jim R. Wilson
  • [HBASE-22610] - [BucketCache] Rename "hbase.offheapcache.minblocksize"
  • [HBASE-22616] - responseTooXXX logging for Multi should characterize the component ops
  • [HBASE-22618] - added the possibility to load custom cost functions
  • [HBASE-22669] - Add unit tests for org.apache.hadoop.hbase.util.Strings
  • [HBASE-22689] - Line break for fix version in documentation
  • [HBASE-22692] - Rubocop definition is not used in the /bin directory
  • [HBASE-22702] - [Log] 'Group not found for table' is chatty
  • [HBASE-22724] - Add a emoji on the vote table for pre commit result on github
  • [HBASE-22760] - Stop/Resume Snapshot Auto-Cleanup activity with shell command
  • [HBASE-22762] - Print the delta between phases in the split/merge/compact/flush transaction journals
  • [HBASE-22785] - Reduce number of Checkstyle issues in client exceptions
  • [HBASE-22786] - Fix Checkstyle issues in tests of hbase-client
  • [HBASE-22804] - Provide an API to get list of successful regions and total expected regions in Canary
  • [HBASE-22810] - Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
  • [HBASE-22828] - Log a region close journal
  • [HBASE-22844] - Fix Checkstyle issues in client snapshot exceptions
  • [HBASE-22872] - Don't create normalization plan unnecesarily when split and merge both are disabled
  • [HBASE-22874] - Define a public interface for Canary and move existing implementation to LimitedPrivate
  • [HBASE-22880] - [Backport] HBASE-22871 to branch-1
  • [HBASE-22890] - Verify the files when RegionServer is starting and BucketCache is in file mode
  • [HBASE-22912] - [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads
  • [HBASE-22930] - Set unique name to longCompactions/shortCompactions threads
  • [HBASE-22975] - Add read and write QPS metrics at server level and table level
  • [HBASE-23038] - Provide consistent and clear logging about disabling chores
  • [HBASE-23058] - Should be "Column Family Name" in table.jsp
  • [HBASE-23116] - LoadBalancer should log table name when balancing per table

Test

  • [HBASE-20667] - Rename TestGlobalThrottler to TestReplicationGlobalThrottler
  • [HBASE-21272] - Re-add assertions for RS Group admin tests
  • [HBASE-21952] - Test Failure: TestClientOperationInterrupt.testInterrupt50Percent
  • [HBASE-22051] - Expect values are hard-coded in the verifications of TestRSGroupsBasics
  • [HBASE-22450] - Port TestStoreScannerClosure from HBASE-22072
  • [HBASE-22545] - TestLogLevel broken
  • [HBASE-22615] - Make TestChoreService more robust to timing
  • [HBASE-22725] - Remove all remaining javadoc warnings

Wish

  • [HBASE-22833] - MultiRowRangeFilter should provide a method for creating a filter which is functionally equivalent to multiple prefix filters

Task

  • [HBASE-19230] - Write up fixVersion policy from dev discussion in refguide
  • [HBASE-19858] - Backport HBASE-14061 (Support CF-level Storage Policy) to branch-1
  • [HBASE-20486] - Change default throughput controller to PressureAwareThroughputController in branch-1
  • [HBASE-20493] - Port HBASE-19994 (Create a new class for RPC throttling exception, make it retryable) to branch-1
  • [HBASE-20501] - Change the Hadoop minimum version to 2.7.1
  • [HBASE-21606] - Document use of the meta table load metrics added in HBASE-19722
  • [HBASE-21791] - Upgrade thrift dependency to 0.12.0
  • [HBASE-21826] - Rebase 1.5.0 CHANGES on branch-1.4 at release 1.4.9
  • [HBASE-21969] - Improve the update of destination rsgroup of RSGroupInfoManagerImpl#moveTables()
  • [HBASE-22044] - ByteBufferUtils should not be IA.Public API
  • [HBASE-22409] - update branch-1 ref guide section on prereqs
  • [HBASE-22444] - Backport HBASE-19954 to branch-1
  • [HBASE-22556] - [DOCS] Backport HBASE-15557 to branch-1 and branch-2
  • [HBASE-22911] - fewer concurrent github PR builds
  • [HBASE-22913] - Use Hadoop label for nightly builds
  • [HBASE-23023] - upgrade shellcheck used to test in nightly and precommit
  • [HBASE-23024] - Replace Exception.initCause() with Constructor args

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.