Release Notes - Bigtop - Version 0.5.0 - HTML format

Sub-task

  • [BIGTOP-612] - Add Crunch to Bigtop
  • [BIGTOP-692] - Bump Whirr to 0.8.1
  • [BIGTOP-695] - Make variables used in flume build/install scripts consistent
  • [BIGTOP-696] - Make variables used in oozie build/install scripts consistent
  • [BIGTOP-697] - Make variables used in zookeeper build/install scripts consistent
  • [BIGTOP-700] - Make variables used in whirr build/install scripts consistent
  • [BIGTOP-702] - Make variables used in datafu build/install scripts consistent
  • [BIGTOP-703] - Make variables used in jsvc build/install scripts consistent
  • [BIGTOP-704] - Make variables used in pig build/install scripts consistent
  • [BIGTOP-706] - Make variables used in hive build/install scripts consistent
  • [BIGTOP-707] - Make variables used in tomcat build/install scripts consistent
  • [BIGTOP-711] - update hadoop version to 2.0.2-alpha
  • [BIGTOP-714] - HDFS Fuse build broken for 0.23.3
  • [BIGTOP-733] - Bump Hue to 2.1.0
  • [BIGTOP-735] - Upgrade to pull Solr 4.0 when it's officially released
  • [BIGTOP-736] - When Solr is integrated, we'll need some tests.
  • [BIGTOP-744] - add package test for Solr
  • [BIGTOP-750] - Bump Zookeeper to 3.4.5
  • [BIGTOP-751] - Bump HBase to 0.94.2
  • [BIGTOP-752] - Bump Sqoop to 1.4.2
  • [BIGTOP-753] - Bump Flume to 1.3.0
  • [BIGTOP-765] - Create deployment code for Solr
  • [BIGTOP-791] - Bump Oozie to 3.3.0

Bug

  • [BIGTOP-106] - review and cleanup install/unistall/upgrade time dependencies
  • [BIGTOP-222] - some man pages in /usr/share/doc/flume
  • [BIGTOP-226] - everything in /usr/lib/pig is set to 755. Non executable files should just be 644
  • [BIGTOP-229] - pig package install an alternative with an inconsitent name
  • [BIGTOP-307] - README documentation is inaccurate
  • [BIGTOP-335] - sqoop-metastore service fails to start because it ends up using /usr/lib/hadoop/bin/hadoop
  • [BIGTOP-347] - we need to get rid of workaround for HDFS-1943 in hadoop init.d scripts
  • [BIGTOP-385] - new artifacts in Hadoop 0.23 need to be properly packaged
  • [BIGTOP-395] - flume-ng init.d scripts seem to be unhappy on Debian
  • [BIGTOP-471] - hive server setup after installation fails on creating /var/lock/subsys directory
  • [BIGTOP-520] - HBase Debian builds run do-component-build twice
  • [BIGTOP-525] - make sure yarn.dispatcher.exit-on-error is set to true
  • [BIGTOP-542] - Issue when installing the new zkfc
  • [BIGTOP-591] - libhdfs0-dev is missing files
  • [BIGTOP-634] - Weird output when doing an init action on oozie when it is already running
  • [BIGTOP-648] - hbase-thrift cannot be started properly
  • [BIGTOP-654] - Set nproc for HDFS user
  • [BIGTOP-658] - Move hbase dependency in hive out of the main package
  • [BIGTOP-662] - cannot start hue-server
  • [BIGTOP-663] - cannot start sqoop-metastore in Ubuntu precise
  • [BIGTOP-664] - hive-metastore script has error when execute 'service hive-metastore status' in Ubuntu precise
  • [BIGTOP-667] - start hadoop-httpfs in SLES return 3 instead of 0
  • [BIGTOP-669] - Add DataFu to Bigtop distribution
  • [BIGTOP-676] - Update HBase version to 0.94.1-rc0
  • [BIGTOP-679] - fuse_dfs binary has moved due to HDFS-3537
  • [BIGTOP-684] - copyright header in bigtop-packages/src/common/hadoop/conf.secure/core-site.xml is in wrong place
  • [BIGTOP-686] - Apache Flume 1.2.0 went out so we should upgrade to it
  • [BIGTOP-689] - Oozie use of /var/tmp in some CentOS versions create issues
  • [BIGTOP-691] - flume gets killed too fast when the service is asked to stop
  • [BIGTOP-694] - Some HBase Scripts Need To Be Excluded From Packaging
  • [BIGTOP-699] - hadoop-fuse-dfs fails to find libjvm.so when JAVA_HOME happens to be a symlink
  • [BIGTOP-701] - Bigtop Hadoop package does not include bash tab completion
  • [BIGTOP-708] - modify Hive integration test to depends on the correct jar file
  • [BIGTOP-721] - improve the package daemon status check, check twice by some delay if status doesn't match expected value
  • [BIGTOP-725] - package service test TestServices should only do config once for a group of services
  • [BIGTOP-734] - Proposed fixes to Hive logging configuration
  • [BIGTOP-745] - fix incorrect javadocs in test framework
  • [BIGTOP-754] - libhdfs0-dev package is empty
  • [BIGTOP-755] - YARN_HOME is now HADOOP_YARN_HOME
  • [BIGTOP-758] - [Pig] Override hadoop-core and hadoop-test artifact versions only if Hadoop < 2.x
  • [BIGTOP-759] - Flume do-component-build should set hbase.version according to BOM
  • [BIGTOP-762] - Updates for building Flume versions >= 1.2
  • [BIGTOP-768] - Bump the version of Groovy to 1.8
  • [BIGTOP-774] - solr build fails on Debian
  • [BIGTOP-775] - In Groovy 1.8 the behavior of a getters of static fields changed slightly
  • [BIGTOP-776] - A few updates to Oozie related packaging code to accomodate Oozie's TLP status
  • [BIGTOP-778] - move Solr app to under /solr in Tomcat deployment and provide more knobs
  • [BIGTOP-780] - cannot install hive-server in SLES
  • [BIGTOP-781] - Bigtop packages seem to have a bunch of parasitic provides
  • [BIGTOP-782] - 'service hue status' still show 'failed' after hue is started in SLES
  • [BIGTOP-783] - 'service hue stop' cannot stop the hue service in SLES
  • [BIGTOP-785] - clean up build-time dependencies in our packages
  • [BIGTOP-788] - Oozie RPM build error due to lack of usr/lib/oozie/{libserver,oozie-server,libext}
  • [BIGTOP-789] - remove shadow-utils dependency from zookeeper
  • [BIGTOP-793] - add crunch integration tests
  • [BIGTOP-796] - hive install tries to tickle HDFS and it shouldn't
  • [BIGTOP-797] - provide a way to 'rsync' files into HDFS during puppet deployment
  • [BIGTOP-798] - introduce a fatjar collection of all the Bigtop integration tests and all their dependencies
  • [BIGTOP-800] - BIGTOP_JSVC is lacking ARCHIVE designation
  • [BIGTOP-801] - prioritize jdk over jre in bigtop-utils javadetection

New Feature

  • [BIGTOP-42] - Start doing testpatch-like patch validation
  • [BIGTOP-685] - provide a way to specify the parameters expected by a test
  • [BIGTOP-687] - Support YARN as a subproject of Hadoop

Improvement

  • [BIGTOP-351] - Make HBASE_HEAPSIZE manageable via puppet
  • [BIGTOP-517] - Flume packaging should use the term "agent" instead of "node".
  • [BIGTOP-536] - provide suitable title in the <title> tag of the main home page for bigtop-incubating
  • [BIGTOP-576] - Minor discrepancy in naming of logs/users vs. base log directories
  • [BIGTOP-670] - provide a link to our Jenkins server from the Bigtop's project page
  • [BIGTOP-680] - Write util function to capture test input parameters
  • [BIGTOP-688] - improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
  • [BIGTOP-718] - Update wiki for component requirements
  • [BIGTOP-726] - make it possible to run HiveBulkScriptExecutor scripts selectively
  • [BIGTOP-731] - Reorganize Hadoop tests
  • [BIGTOP-742] - tighten permissions on hadoop services home directories
  • [BIGTOP-757] - Install ZooKeeper jars into the local Maven cache
  • [BIGTOP-772] - Remove all references to the incubation from the trunk of Bigtop
  • [BIGTOP-777] - separate the Package Tests into service tests, file content tests, and dependency tests
  • [BIGTOP-787] - Surface all possible options in /etc/default/flume-node
  • [BIGTOP-799] - create an itest method for unpacking all the resources into the HDFS or local filesystem
  • [BIGTOP-802] - Add rollback option to DataNode service script
  • [BIGTOP-806] - now that MAPREDUCE-3916 is fixed we should enable webproxy

Wish

  • [BIGTOP-67] - Resize bigtop logo in header of website

Task

  • [BIGTOP-406] - Enable openSuse VM on bigtop01
  • [BIGTOP-439] - we need to review package test manifest for the Bigtop 0.4.0 release

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.