Release Notes - HBase - Version 0.98.1 - HTML format

Sub-task

  • [HBASE-5175] - Add DoubleColumnInterpreter
  • [HBASE-9966] - Create IntegrationTest for Online Bloom Filter Change
  • [HBASE-10169] - Batch coprocessor
  • [HBASE-10526] - Using Cell instead of KeyValue in HFileOutputFormat
  • [HBASE-10651] - Fix incorrect handling of IE that restores current thread's interrupt status within while/for loops in Replication
  • [HBASE-10697] - Convert TestSimpleTotalOrderPartitioner to junit4 test
  • [HBASE-10717] - TestFSHDFSUtils#testIsSameHdfs fails with IllegalArgumentException running against hadoop 2.3
  • [HBASE-10723] - Convert TestExplicitColumnTracker to junit4 test
  • [HBASE-10741] - Deprecate HTablePool and HTableFactory

Bug

  • [HBASE-6642] - enable_all,disable_all,drop_all can call "list" command with regex directly.
  • [HBASE-8304] - Bulkload fails to remove files if fs.default.name / fs.defaultFS is configured without default port
  • [HBASE-8803] - region_mover.rb should move multiple regions at a time
  • [HBASE-9294] - NPE in /rs-status during RS shutdown
  • [HBASE-9708] - Improve Snapshot Name Error Message
  • [HBASE-9719] - Premptive Call Me Maybe HBase
  • [HBASE-9721] - RegionServer should not accept regionOpen RPC intended for another(previous) server
  • [HBASE-9778] - Add hint to ExplicitColumnTracker to avoid seeking
  • [HBASE-9990] - HTable uses the conf for each "newCaller"
  • [HBASE-10080] - Unnecessary call to locateRegion when creating an HTable instance
  • [HBASE-10097] - Remove a region name string creation in HRegion#nextInternal
  • [HBASE-10117] - Avoid synchronization in HRegionScannerImpl.isFilterDone
  • [HBASE-10313] - Duplicate servlet-api jars in hbase 0.96.0
  • [HBASE-10413] - Tablesplit.getLength returns 0
  • [HBASE-10432] - Rpc retries non-recoverable error
  • [HBASE-10436] - restore regionserver lists removed from hbase 0.96+ jmx
  • [HBASE-10451] - Enable back Tag compression on HFiles
  • [HBASE-10452] - Fix potential bugs in exception handlers
  • [HBASE-10453] - PrefixTree encoding for tags can honor HCD#shouldCompressTags()
  • [HBASE-10476] - HBase Master log grows very fast after stopped hadoop (due to connection exception)
  • [HBASE-10481] - API Compatibility JDiff script does not properly handle arguments in reverse order
  • [HBASE-10482] - ReplicationSyncUp doesn't clean up its ZK, needed for tests
  • [HBASE-10485] - PrefixFilter#filterKeyValue() should perform filtering on row key
  • [HBASE-10486] - ProtobufUtil Append & Increment deserialization lost cell level timestamp
  • [HBASE-10493] - InclusiveStopFilter#filterKeyValue() should perform filtering on row key
  • [HBASE-10495] - upgrade script is printing usage two times with help option.
  • [HBASE-10500] - Some tools OOM when BucketCache is enabled
  • [HBASE-10501] - Improve IncreasingToUpperBoundRegionSplitPolicy to avoid too many regions
  • [HBASE-10514] - Forward port HBASE-10466, possible data loss when failed flushes
  • [HBASE-10527] - TestTokenAuthentication fails with the IBM JDK
  • [HBASE-10534] - Rowkey in TsvImporterTextMapper initializing with wrong length
  • [HBASE-10537] - Let the ExportSnapshot mapper fail and retry on error
  • [HBASE-10539] - HRegion.addAndGetGlobalMemstoreSize returns previous size
  • [HBASE-10545] - RS Hangs waiting on region to close on shutdown; has to timeout before can go down
  • [HBASE-10546] - Two scanner objects are open for each hbase map task but only one scanner object is closed
  • [HBASE-10547] - TestFixedLengthWrapper#testReadWrite occasionally fails with the IBM JDK
  • [HBASE-10548] - Correct commons-math dependency version
  • [HBASE-10552] - HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes.
  • [HBASE-10556] - Possible data loss due to non-handled DroppedSnapshotException for user-triggered flush from client/shell
  • [HBASE-10563] - Set name for FlushHandler thread
  • [HBASE-10564] - HRegionServer.nextLong should be removed since it's not used anywhere, or should be used somewhere it meant to
  • [HBASE-10565] - FavoredNodesPlan accidentally uses an internal Netty type
  • [HBASE-10567] - Add overwrite manifest option to ExportSnapshot
  • [HBASE-10575] - ReplicationSource thread can't be terminated if it runs into the loop to contact peer's zk ensemble and fails continuously
  • [HBASE-10579] - [Documentation]: ExportSnapshot tool package incorrectly documented
  • [HBASE-10580] - IntegrationTestingUtility#restoreCluster leak resource when running in a mini cluster mode
  • [HBASE-10581] - ACL znode are left without PBed during upgrading hbase0.94* to hbase0.96+
  • [HBASE-10582] - 0.94->0.96 Upgrade: ACL can't be repopulated when ACL table contains row for table '-ROOT' or '.META.'
  • [HBASE-10585] - Avoid early creation of Node objects in LRUDictionary.BidirectionalLRUMap
  • [HBASE-10586] - hadoop2-compat IPC metric registred twice
  • [HBASE-10587] - Master metrics clusterRequests is wrong
  • [HBASE-10594] - Speed up TestRestoreSnapshotFromClient
  • [HBASE-10598] - Written data can not be read out because MemStore#timeRangeTracker might be updated concurrently
  • [HBASE-10600] - HTable#batch() should perform validation on empty Put
  • [HBASE-10604] - Fix parseArgs javadoc
  • [HBASE-10608] - Acquire the FS Delegation Token for Secure ExportSnapshot
  • [HBASE-10614] - Master could not be stopped
  • [HBASE-10618] - User should not be allowed to disable/drop visibility labels table
  • [HBASE-10621] - Unable to grant user permission to namespace
  • [HBASE-10622] - Improve log and Exceptions in Export Snapshot
  • [HBASE-10624] - Fix 2 new findbugs warnings introduced by HBASE-10598
  • [HBASE-10627] - A logic mistake in HRegionServer isHealthy
  • [HBASE-10631] - Avoid extra seek on FileLink open
  • [HBASE-10632] - Region lost in limbo after ArrayIndexOutOfBoundsException during assignment
  • [HBASE-10639] - Unload script displays wrong counts (off by one) when unloading regions
  • [HBASE-10644] - TestSecureExportSnapshot#testExportFileSystemState fails on hadoop-1
  • [HBASE-10656] - high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
  • [HBASE-10660] - MR over snapshots can OOM when alternative blockcache is enabled
  • [HBASE-10662] - RegionScanner is never closed if the region has been moved-out or re-opened when performing scan request
  • [HBASE-10665] - TestCompaction and TestCompactionWithCoprocessor run too long
  • [HBASE-10666] - TestMasterCoprocessorExceptionWithAbort hangs at shutdown
  • [HBASE-10668] - TestExportSnapshot runs too long
  • [HBASE-10669] - [hbck tool] Usage is wrong for hbck tool for -sidelineCorruptHfiles option
  • [HBASE-10675] - IntegrationTestIngestWithACL should allow User to be passed as Parameter
  • [HBASE-10677] - boundaries check in hbck throwing IllegalArgumentException
  • [HBASE-10679] - Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region
  • [HBASE-10682] - region_mover.rb throws "can't convert nil into String" for regions moved
  • [HBASE-10685] - [WINDOWS] TestKeyStoreKeyProvider fails on windows
  • [HBASE-10686] - [WINDOWS] TestStripeStoreFileManager fails on windows
  • [HBASE-10694] - TableSkewCostFunction#cost() casts integral division result to double
  • [HBASE-10706] - Disable writeToWal in tests where possible
  • [HBASE-10716] - [Configuration]: hbase.regionserver.region.split.policy should be part of hbase-default.xml
  • [HBASE-10718] - TestHLogSplit fails when it sets a KV size to be negative
  • [HBASE-10726] - Fix java.lang.ArrayIndexOutOfBoundsException in StochasticLoadBalancer$LocalityBasedCandidateGenerator
  • [HBASE-10731] - Fix environment variables typos in scripts
  • [HBASE-10735] - [WINDOWS] Set -XX:MaxPermSize for unit tests
  • [HBASE-10736] - Fix Javadoc warnings introduced in HBASE-10169
  • [HBASE-10737] - HConnectionImplementation should stop RpcClient on close
  • [HBASE-10738] - AssignmentManager should shut down executors on stop
  • [HBASE-10739] - RS web UI NPE if master shuts down sooner
  • [HBASE-10745] - Access ShutdownHook#fsShutdownHooks should be synchronized
  • [HBASE-10749] - CellComparator.compareStatic() compares type wrongly
  • [HBASE-10751] - TestHRegion testWritesWhileScanning occasional fail since HBASE-10514 went in
  • [HBASE-10760] - Wrong methods' names in ClusterLoadState class
  • [HBASE-10762] - clone_snapshot doesn't check for missing namespace
  • [HBASE-10763] - Backport HBASE-10549(When there is a hole, LoadIncrementalHFiles will hang in an infinite loop.) to 0.98
  • [HBASE-10766] - SnapshotCleaner allows to delete referenced files
  • [HBASE-10770] - Don't exit from the Canary daemon mode if no regions are present
  • [HBASE-10793] - AuthFailed as a valid zookeeper state
  • [HBASE-10799] - [WINDOWS] TestImportTSVWithVisibilityLabels.testBulkOutputWithTsvImporterTextMapper fails on windows
  • [HBASE-10802] - CellComparator.compareStaticIgnoreMvccVersion compares type wrongly
  • [HBASE-10804] - Add a validations step to ExportSnapshot
  • [HBASE-10805] - Speed up KeyValueHeap.next() a bit
  • [HBASE-10806] - Two protos missing in hbase-protocol/pom.xml
  • [HBASE-10809] - HBaseAdmin#deleteTable fails when META region happen to move around same time
  • [HBASE-10825] - Add copy-from option to ExportSnapshot
  • [HBASE-10829] - Flush is skipped after log replay if the last recovered edits file is skipped
  • [HBASE-10833] - Region assignment may fail during cluster start up
  • [HBASE-10838] - Insufficient AccessController covering permission check
  • [HBASE-10862] - Update config field names in hbase-default.xml description for hbase.hregion.memstore.block.multiplier
  • [HBASE-10863] - Scan doesn't return rows for user who has authorization by visibility label in secure deployment

New Feature

  • [HBASE-8751] - Enable peer cluster to choose/change the ColumnFamilies/Tables it really want to replicate from a source cluster

Improvement

  • [HBASE-7849] - Provide administrative limits around bulkloads of files into a single region
  • [HBASE-8076] - add better doc for HBaseAdmin#offline API.
  • [HBASE-8604] - improve reporting of incorrect peer address in replication
  • [HBASE-9501] - Provide throttling for replication
  • [HBASE-9999] - Add support for small reverse scan
  • [HBASE-10389] - Add namespace help info in table related shell commands
  • [HBASE-10419] - Add multiget support to PerformanceEvaluation
  • [HBASE-10487] - Avoid allocating new KeyValue and according bytes-copying for appended kvs which don't have existing values
  • [HBASE-10498] - Add new APIs to load balancer interface
  • [HBASE-10511] - Add latency percentiles on PerformanceEvaluation
  • [HBASE-10518] - DirectMemoryUtils.getDirectMemoryUsage spams when none is configured
  • [HBASE-10570] - Allow overrides of Surefire secondPartForkMode and testFailureIgnore
  • [HBASE-10589] - Reduce unnecessary TestRowProcessorEndpoint resource usage
  • [HBASE-10592] - Refactor PerformanceEvaluation tool
  • [HBASE-10597] - IOEngine#read() should return the number of bytes transferred
  • [HBASE-10638] - Improve error message when there is no region server available for move
  • [HBASE-10678] - Make verifyrep tool implement toolrunner
  • [HBASE-10693] - Correct declarations of Atomic* fields from 'volatile' to 'final'
  • [HBASE-10744] - AM#CloseRegion no need to retry on FailedServerException
  • [HBASE-10769] - hbase/bin/hbase-cleanup.sh has wrong usage string
  • [HBASE-10797] - Add support for -h and --help to rolling_restart.sh and fix the usage string output
  • [HBASE-10812] - Backport HBASE-9426(Make custom distributed barrier procedure pluggable) to 0.98.x
  • [HBASE-10887] - tidy ThriftUtilities format

Test

  • [HBASE-10543] - Two rare test failures with TestLogsCleaner and TestSplitLogWorker
  • [HBASE-10649] - TestMasterMetrics fails occasionally
  • [HBASE-10764] - TestLoadIncrementalHFilesSplitRecovery#testBulkLoadPhaseFailure takes too long
  • [HBASE-10767] - Load balancer may interfere with tests in TestHBaseFsck
  • [HBASE-10782] - Hadoop2 MR tests fail occasionally because of mapreduce.jobhistory.address is no set in job conf
  • [HBASE-10828] - TestRegionObserverInterface#testHBase3583 should wait for all regions to be assigned
  • [HBASE-10852] - TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
  • [HBASE-10868] - TestAtomicOperation should close HRegion instance after each subtest

Task

  • [HBASE-10612] - Remove unnecessary dependency on org.eclipse.jdt:core
  • [HBASE-10670] - HBaseFsck#connect() should use new connection
  • [HBASE-10700] - IntegrationTestWithCellVisibilityLoadAndVerify should allow current user to be the admin
  • [HBASE-10740] - Upgrade zookeeper to 3.4.6 release
  • [HBASE-10786] - If snapshot verification fails with 'Regions moved', the message should contain the name of region causing the failure

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.