Release Notes - HBase - Version 1.3.3 - HTML format

Sub-task

  • [HBASE-16848] - Usage for show_peer_tableCFs command doesn't include peer
  • [HBASE-18786] - FileNotFoundException should not be silently handled for primary region replicas
  • [HBASE-20316] - Backport HBASE-20229 "ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on" to branch-1
  • [HBASE-20317] - Backport HBASE-20261 "Table page (table.jsp) in Master UI does not show replicaIds for hbase meta table" to branch-1
  • [HBASE-20557] - Backport HBASE-17215 to branch-1
  • [HBASE-20558] - Backport HBASE-17854 to branch-1
  • [HBASE-20559] - Backport HBASE-18083 to branch-1
  • [HBASE-20647] - Backport HBASE-20616 "TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state" to branch-1
  • [HBASE-20974] - Backport HBASE-20583 (SplitLogWorker should handle FileNotFoundException when split a wal) to branch-1
  • [HBASE-21060] - fix dead store in SecureBulkLoadEndpoint
  • [HBASE-21061] - fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
  • [HBASE-21189] - flaky job should gather machine stats
  • [HBASE-21190] - Log files and count of entries in each as we load from the MasterProcWAL store
  • [HBASE-21347] - Backport HBASE-21200 "Memstore flush doesn't finish because of seekToPreviousRow() in memstore scanner." to branch-1

Bug

  • [HBASE-15291] - FileSystem not closed in secure bulkLoad
  • [HBASE-16091] - Canary takes lot more time when there are delete markers in the table
  • [HBASE-16576] - Shell add_peer doesn't allow setting cluster_key for custom endpoints
  • [HBASE-16910] - Avoid NPE when starting StochasticLoadBalancer
  • [HBASE-17072] - CPU usage starts to climb up to 90-100% when using G1GC; purge ThreadLocal usage
  • [HBASE-17118] - StoreScanner leaked in KeyValueHeap
  • [HBASE-17565] - StochasticLoadBalancer may incorrectly skip balancing due to skewed multiplier sum
  • [HBASE-17631] - Canary interval too low
  • [HBASE-17798] - RpcServer.Listener.Reader can abort due to CancelledKeyException
  • [HBASE-17816] - HRegion#mutateRowWithLocks should update writeRequestCount metric
  • [HBASE-17861] - Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3
  • [HBASE-17885] - Backport HBASE-15871 to branch-1
  • [HBASE-17930] - Avoid using Canary.sniff in HBaseTestingUtility
  • [HBASE-17991] - Add more details about compaction queue on /dump
  • [HBASE-18058] - Zookeeper retry sleep time should have an upper limit
  • [HBASE-18111] - Replication stuck when cluster connection is closed
  • [HBASE-18451] - PeriodicMemstoreFlusher should inspect the queue before adding a delayed flush request
  • [HBASE-18512] - Region Server will abort with IllegalStateException if HDFS umask has limited scope
  • [HBASE-18762] - Canary sink type cast error
  • [HBASE-18830] - TestCanaryTool does not check Canary monitor's error code
  • [HBASE-18842] - The hbase shell clone_snaphost command returns bad error message
  • [HBASE-18862] - backport HBASE-15109 to branch-1.2,branch-1.3
  • [HBASE-19163] - "Maximum lock count exceeded" from region server's batch processing
  • [HBASE-19343] - Restore snapshot makes split parent region online
  • [HBASE-19364] - Truncate_preserve fails with table when replica region > 1
  • [HBASE-19377] - Compatibility checker complaining about hash collisions
  • [HBASE-19391] - Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory
  • [HBASE-19418] - RANGE_OF_DELAY in PeriodicMemstoreFlusher should be configurable.
  • [HBASE-19435] - Reopen Files for ClosedChannelException in BucketCache
  • [HBASE-19458] - Allow building HBase 1.3.x against Hadoop 2.8
  • [HBASE-19551] - hbck -boundaries doesn't work correctly
  • [HBASE-19553] - Old replica regions should be cleared from AM memory after primary region split or merge
  • [HBASE-19816] - Replication sink list is not updated on UnknownHostException
  • [HBASE-19905] - ReplicationSyncUp tool will not exit if a peer replication is disabled
  • [HBASE-19924] - hbase rpc throttling does not work for multi() with request count rater.
  • [HBASE-20004] - Client is not able to execute REST queries in a secure cluster
  • [HBASE-20068] - Hadoopcheck project health check uses default maven repo instead of yetus managed ones
  • [HBASE-20141] - Fix TooManyFiles exception when RefreshingChannels in FileIOEngine
  • [HBASE-20230] - Incorrrect log message in RSRpcService
  • [HBASE-20231] - Not able to delete column family from a row using RemoteHTable
  • [HBASE-20292] - Wrong URLs in the descriptions for update_all_config and update_config commands in shell
  • [HBASE-20302] - CatalogJanitor should log the reason why it is disabled
  • [HBASE-20322] - CME in StoreScanner causes region server crash
  • [HBASE-20335] - nightly jobs no longer contain machine information
  • [HBASE-20364] - nightly job gives old results or no results for stages that timeout on SCM
  • [HBASE-20404] - Ugly cleanerchore complaint that dir is not empty
  • [HBASE-20517] - Fix PerformanceEvaluation 'column' parameter
  • [HBASE-20554] - "WALs outstanding" message from CleanerChore is noisy
  • [HBASE-20571] - JMXJsonServlet generates invalid JSON if it has NaN in metrics
  • [HBASE-20575] - Fail to config COMPACTION_ENABLED by hbase shell
  • [HBASE-20590] - REST Java client is not able to negotiate with the server in the secure mode
  • [HBASE-20591] - nightly job doesn't respect maven options
  • [HBASE-20597] - Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
  • [HBASE-20604] - ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled
  • [HBASE-20638] - nightly source artifact testing should fail the stage if it's going to report an error on jira
  • [HBASE-20664] - Variable shared across multiple threads
  • [HBASE-20669] - [findbugs] autoboxing to parse primitive
  • [HBASE-20689] - Docker fails to install rubocop for precommit
  • [HBASE-20697] - Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations()
  • [HBASE-20704] - Sometimes some compacted storefiles are not archived on region close
  • [HBASE-20723] - Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
  • [HBASE-20732] - Shutdown scan pool when master is stopped.
  • [HBASE-20766] - Verify Replication Tool Has Typo "remove cluster"
  • [HBASE-20769] - getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
  • [HBASE-20770] - WAL cleaner logs way too much; gets clogged when lots of work to do
  • [HBASE-20771] - PUT operation fail with "No server address listed in hbase:meta for region xxxxx"
  • [HBASE-20785] - NPE getting metrics in PE testing scans
  • [HBASE-20808] - Wrong shutdown order between Chores and ChoreService
  • [HBASE-20866] - HBase 1.x scan performance degradation compared to 0.98 version
  • [HBASE-20889] - PE scan is failing with NullPointerException
  • [HBASE-20890] - PE filterScan seems to be stuck forever
  • [HBASE-20895] - NPE in RpcServer#readAndProcess
  • [HBASE-20908] - Infinite loop on regionserver if region replica are reduced
  • [HBASE-20940] - HStore.cansplit should not allow split to happen if it has references
  • [HBASE-20997] - rebuildUserRegions() does not build ReplicaMapping during master switchover
  • [HBASE-21007] - Memory leak in HBase rest server
  • [HBASE-21030] - Correct javadoc for append operation
  • [HBASE-21042] - processor.getRowsToLock() always assumes there is some row being locked in HRegion#processRowsWithLocks
  • [HBASE-21047] - Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
  • [HBASE-21058] - Nightly tests for branches 1 fail to build ref guide
  • [HBASE-21067] - Backport HBASE-17519 (Rollback the removed cells) to branch-1.3
  • [HBASE-21069] - NPE in StoreScanner.updateReaders causes RS to crash
  • [HBASE-21074] - JDK7 branches need to pass "-Dhttps.protocols=TLSv1.2" to maven when building
  • [HBASE-21088] - HStoreFile should be closed in HStore#hasReferences
  • [HBASE-21104] - client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
  • [HBASE-21158] - Empty qualifier cell should not be returned if it does not match QualifierFilter
  • [HBASE-21166] - Creating a CoprocessorHConnection re-retrieves the cluster id from ZK
  • [HBASE-21173] - Remove the duplicate HRegion#close in TestHRegion
  • [HBASE-21179] - Fix the number of actions in responseTooSlow log
  • [HBASE-21203] - TestZKMainServer#testCommandLineWorks won't pass with default 4lw whitelist
  • [HBASE-21212] - Wrong flush time when update flush metric
  • [HBASE-21228] - Memory leak since AbstractFSWAL caches Thread object and never clean later
  • [HBASE-21266] - Not running balancer because processing dead regionservers, but empty dead rs list
  • [HBASE-21275] - Thrift Server (branch 1 fix) -> Disable TRACE HTTP method for thrift http server (branch 1 only)
  • [HBASE-21355] - HStore's storeSize is calculated repeatedly which causing the confusing region split
  • [HBASE-21357] - RS should abort if OOM in Reader thread
  • [HBASE-21359] - Fix build problem against Hadoop 2.8.5
  • [HBASE-21387] - Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files
  • [HBASE-21417] - Pre commit build is broken due to surefire plugin crashes
  • [HBASE-21439] - StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions
  • [HBASE-21445] - CopyTable by bulkload will write hfile into yarn's HDFS
  • [HBASE-21546] - ConnectException in TestThriftHttpServer
  • [HBASE-21553] - schedLock not released in MasterProcedureScheduler
  • [HBASE-21582] - If call HBaseAdmin#snapshotAsync but forget call isSnapshotFinished, then SnapshotHFileCleaner will skip to run every time

New Feature

  • [HBASE-15134] - Add visibility into Flush and Compaction queues

Improvement

  • [HBASE-15466] - precommit should not run all java goals when given a docs-only patch
  • [HBASE-15529] - Override needBalance in StochasticLoadBalancer
  • [HBASE-15727] - Canary Tool for Zookeeper
  • [HBASE-16399] - Provide an API to get list of failed regions and servername in Canary
  • [HBASE-16947] - Some improvements for DumpReplicationQueues tool
  • [HBASE-17205] - Add a metric for the duration of region in transition
  • [HBASE-17437] - Support specifying a WAL directory outside of the root directory
  • [HBASE-17731] - Fractional latency reporting in MultiThreadedAction
  • [HBASE-17912] - Avoid major compactions on region server startup
  • [HBASE-17924] - Consider sorting the row order when processing multi() ops before taking rowlocks
  • [HBASE-17959] - Canary timeout should be configurable on a per-table basis
  • [HBASE-18248] - Warn if monitored RPC task has been tied up beyond a configurable threshold
  • [HBASE-20047] - AuthenticationTokenIdentifier should provide a toString
  • [HBASE-20307] - LoadTestTool prints too much zookeeper logging
  • [HBASE-20352] - [Chore] Backport HBASE-18309 to branch-1
  • [HBASE-20379] - shadedjars yetus plugin should add a footer link
  • [HBASE-20387] - flaky infrastructure should work for all branches
  • [HBASE-20401] - Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
  • [HBASE-20469] - Directory used for sidelining old recovered edits files should be made configurable
  • [HBASE-20545] - Improve performance of BaseLoadBalancer.retainAssignment
  • [HBASE-20548] - Master fails to startup on large clusters, refreshing block distribution
  • [HBASE-20605] - Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
  • [HBASE-20651] - Master, prevents hbck or shell command to reassign the split parent region
  • [HBASE-20672] - New metrics ReadRequestRate and WriteRequestRate
  • [HBASE-20701] - too much logging when balancer runs from BaseLoadBalancer
  • [HBASE-20733] - QABot should run checkstyle tests if the checkstyle configs change
  • [HBASE-20734] - Colocate recovered edits directory with hbase.wal.dir
  • [HBASE-20806] - Split style journal for flushes and compactions
  • [HBASE-20826] - Truncate responseInfo attributes on RpcServer WARN messages
  • [HBASE-20858] - port HBASE-20695 to branch-1
  • [HBASE-20930] - MetaScanner.metaScan should use passed variable for meta table name rather than TableName.META_TABLE_NAME
  • [HBASE-20935] - HStore.removeCompactedFiles should log in case it is unable to delete a file
  • [HBASE-21028] - Backport HBASE-18633 to branch-1.3
  • [HBASE-21103] - nightly test cache of yetus install needs to be more thorough in verification
  • [HBASE-21185] - WALPrettyPrinter: Additional useful info to be printed by wal printer tool, for debugability purposes
  • [HBASE-21263] - Mention compression algorithm along with other storefile details
  • [HBASE-21373] - Backport to branch-1, "HBASE-21338 [balancer] If balancer is an ill-fit for cluster size, it gives little indication"
  • [HBASE-21511] - Remove in progress snapshot check in SnapshotFileCache#getUnreferencedFiles
  • [HBASE-21567] - Allow overriding configs starting up the shell

Test

  • [HBASE-20052] - TestRegionOpen#testNonExistentRegionReplica fails due to NPE
  • [HBASE-20505] - PE should support multi column family read and write cases
  • [HBASE-20513] - Collect and emit ScanMetrics in PerformanceEvaluation
  • [HBASE-20619] - TestWeakObjectPool occasionally times out
  • [HBASE-21076] - TestTableResource fails with NPE
  • [HBASE-21138] - Close HRegion instance at the end of every test in TestHRegion

Task

  • [HBASE-17965] - Canary tool should print the regionserver name on failure
  • [HBASE-19475] - Extend backporting strategy in documentation
  • [HBASE-20112] - Include test results from nightly hadoop3 tests in jenkins test results
  • [HBASE-20665] - "Already cached block XXX" message should be DEBUG
  • [HBASE-20884] - Replace usage of our Base64 implementation with java.util.Base64
  • [HBASE-20905] - branch-1 docker build fails
  • [HBASE-20931] - [branch-1] Add -Dhttps.protocols=TLSv1.2 to Maven command line in make_rc.sh
  • [HBASE-20942] - Improve RpcServer TRACE logging
  • [HBASE-21168] - BloomFilterUtil uses hardcoded randomness

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.