Release Notes - Apache HAWQ (Retired) - Version 2.0.0.0-incubating - HTML format

Sub-task

  • [HAWQ-14] - Hive 1.2.1 support
  • [HAWQ-15] - Hbase 1.1.1 support
  • [HAWQ-16] - HDFS 2.7.1 support
  • [HAWQ-17] - YARN 2.7.1 support
  • [HAWQ-18] - Error out database_size function on hcatalog
  • [HAWQ-175] - Update PXF Version to 3.1.0 (Revert back to 3.0.0)
  • [HAWQ-277] - Remove gradle-wrapper.jar and update gradlew
  • [HAWQ-340] - Make getVersion API return JSON format
  • [HAWQ-459] - Enhance Metadata Rest API to support table(s)
  • [HAWQ-465] - Implement stored procedure to return fields metainfo from PXF
  • [HAWQ-539] - Improve code coverage for dispatcher: connection_fail_after_gang_creation& create_cdb_dispath_result_object& dispmgt_concurrent_connect
  • [HAWQ-546] - Client changes to handle \d for Hive tables
  • [HAWQ-548] - Fix wrong unit test case for ip address parsing
  • [HAWQ-572] - Improve code coverage for dispatcher: fail_qe_after_connection & fail_qe_when_do_query & fail_qe_when_begin_parquet_scan
  • [HAWQ-599] - Fix coverity issues
  • [HAWQ-615] - PXF getMetadata endpoint fails if set of items described by passed pattern have unsupported by Hawq Hive objects
  • [HAWQ-625] - Fix build failure on MAC for the fix of HAWQ-462.
  • [HAWQ-778] - Refine hawq register with some sanity checks
  • [HAWQ-784] - Refine hawq register tests
  • [HAWQ-795] - Import apache/orc library for further development
  • [HAWQ-796] - Extend orc library to support hdfs located file read.
  • [HAWQ-797] - Implement reading tools for testing HDFS located orc file usage.
  • [HAWQ-802] - Refactor alter owner installcheck case using new test framework
  • [HAWQ-806] - Add feature test for subplan with new framework
  • [HAWQ-818] - Add order by for aggregate feature test.
  • [HAWQ-828] - Remove deprecated aggregate, groupingsets test cases in installcheck-good
  • [HAWQ-837] - Add python modules into HAWQ code except paramiko
  • [HAWQ-846] - hawq register document
  • [HAWQ-849] - Remove apache orc codebase from hawq
  • [HAWQ-850] - Planner supports refineCachedPlan interface.
  • [HAWQ-869] - Add regression test for less tuple is inserted issue in prepared statement
  • [HAWQ-919] - pxf has some issues on RAT check
  • [HAWQ-1010] - Add the crypto notice in README

Bug

  • [HAWQ-2] - Build failed due to file missing during initial upload
  • [HAWQ-3] - Missing files during initial upload
  • [HAWQ-4] - make[2]: *** [pygresql] Error 1
  • [HAWQ-6] - hcatalog cause gp_persistent_reset_all() error
  • [HAWQ-7] - Hawq init fail without plpython
  • [HAWQ-9] - fix the Makefile in gpopt subdirectory
  • [HAWQ-10] - hcatalog database causes errors for hawq_toolkit.hawq_size_of_database
  • [HAWQ-12] - "Cannot allocate memory" in parquet_compression test in installcheck-good with hawq dbg build
  • [HAWQ-19] - Money type overflow
  • [HAWQ-20] - Error running analyzedb
  • [HAWQ-21] - cdbpersistentstore.h line 440
  • [HAWQ-23] - Building gpfdist error
  • [HAWQ-26] - Cancelling statements failed and caused data loss
  • [HAWQ-32] - Accessing an external table with error table in a INSERT statement would cause segment fault
  • [HAWQ-33] - Change initdb log location to avoid giving write privileges to gpadmin user on parent hawq data dir.
  • [HAWQ-34] - Skip db 'hcatalog' for psql and persistent table
  • [HAWQ-35] - Large tuples in randomly distributed tables would cause crash and generate core
  • [HAWQ-36] - SELECT * FROM pg_locks causes crash and core
  • [HAWQ-37] - Abort transaction didn't rollback while 1 QE was terminated.
  • [HAWQ-39] - Remove some unused hawq init scripts
  • [HAWQ-41] - cannot open more than 262144 append-only table segment files cocurrently
  • [HAWQ-42] - Disk file corrupt will make HAWQ coredump when read-shortcircuit is enabled in hdfs-client.xml
  • [HAWQ-43] - \h CREATE TABLE should include parquet, exclude CO
  • [HAWQ-48] - Change warning log message to display correct numbers.
  • [HAWQ-50] - persistent table bug while drop tablespace concurrent
  • [HAWQ-53] - Avro Union(complex) type containing nullable type and using null as default failed
  • [HAWQ-56] - Non-deterministic header results with "HEADER" option from external table
  • [HAWQ-58] - Query hang when running test_resourcepool_TolerateLimit
  • [HAWQ-59] - Path Manipulation: HdfsAnalyzer.java
  • [HAWQ-60] - Transformation support in gpfdist is disabled
  • [HAWQ-61] - Fix checking table distribution error
  • [HAWQ-62] - Incorrect calculation of locus describing of hash table introduce core dump in OLAP query
  • [HAWQ-64] - Forward NULL issue for ExtractPrincipalFromTicketCache
  • [HAWQ-65] - Logic error in function:ExtractPrincipalFromTicketCache in resourcebrokewr_libyarn_proc.c
  • [HAWQ-67] - Wrong variable type for saving fgetc() return value
  • [HAWQ-68] - Wrong answer when converting hash to random partition table
  • [HAWQ-69] - HAWQ init will fail when HDFS HA enabled and the URL's depth is less than 2
  • [HAWQ-70] - INSERT INTO foo SELECT * FROM foo returned successful, but the data of foo was not as expected.
  • [HAWQ-71] - Cannot query pg_locks
  • [HAWQ-73] - “create resource queue“ will throw ”Fail to unregister in HAWQ resource manager because of RPC error"
  • [HAWQ-74] - create resource queue failed
  • [HAWQ-75] - When run in yarn mode, the server will crash in schedHost function
  • [HAWQ-78] - HAWQ parallel build(using 'make -j8') failed
  • [HAWQ-83] - Rename resource manager's GUC names in hawq-site.xml
  • [HAWQ-87] - Fix make package of pljava and pgcrypto
  • [HAWQ-89] - format the explain analyze results
  • [HAWQ-90] - gpmigrator crashed because pg_resqueue system table changed.
  • [HAWQ-92] - UDF could not serialize unrecognized node
  • [HAWQ-93] - Remove the test case for InputFormat quicklz.
  • [HAWQ-94] - Storage error when insert a large tuple
  • [HAWQ-95] - Resource broker doesn't check progress if hawq failed to register in Hadoop Yarn
  • [HAWQ-104] - HAWQ RM resource request timeout mechanism does not work
  • [HAWQ-108] - Allocated memory space for saving preferred host information is not freed.
  • [HAWQ-109] - Fix exit code of hawq command line tools
  • [HAWQ-110] - Re-phrase error message for gpcheckhdfs to make it consistent with hawq management utilities
  • [HAWQ-112] - Possible using null pointer in function RB2YARN_registerYARNApplication
  • [HAWQ-113] - NONE mode resource broker minus more pending resource when failed to allocate resource
  • [HAWQ-115] - Format the log of datalocality
  • [HAWQ-116] - default active_statements setting blocks the goh_portals test.
  • [HAWQ-117] - Alter a newly created tablespace table in kerberos mode would cause "ERROR: Append-Only Storage Read could not open segment file"
  • [HAWQ-118] - Hawq treats commented lines as hosts in slaves file
  • [HAWQ-125] - hawq restart should stop with an error message when stop fails
  • [HAWQ-128] - Cleanup pg_resqueue impact in catalog and dump corresponding components
  • [HAWQ-129] - Fix HA and Kerberos Config in yarn-client.xml
  • [HAWQ-130] - Analyzedb utility failing when a table was created using quotes having upper case letters
  • [HAWQ-131] - check hawq_rm_rejectrequest_nseg_limit should use > instead of >=
  • [HAWQ-133] - core when use plpython udf
  • [HAWQ-134] - Multiple session drop table concurrently will cause the table dropped twice successfully
  • [HAWQ-135] - Explain generate serial will cause coredump in optimizer off
  • [HAWQ-136] - Uneven resource allocation request caused queuing queries unable to get even resource allocation
  • [HAWQ-138] - Alter resource quene error when memory set 64mb
  • [HAWQ-139] - Out of memory with 10 concurrent TPC-H workload in YARN mode
  • [HAWQ-141] - memory accessing panic in system test.
  • [HAWQ-142] - Remove fips mode
  • [HAWQ-145] - Memory leak when free the argument instance of resource lease heart-beat thread in dispatcher process
  • [HAWQ-147] - Create Parquet table and insert data in template1 would cause CREATE DATABASE fail.
  • [HAWQ-155] - Out of range access to the hdfs file as scan a large tuple
  • [HAWQ-156] - Unittest hd_work_mgr fails on OSx
  • [HAWQ-159] - Segmentation Fault When Hawq disconnect from libyarn,
  • [HAWQ-160] - Cancel test case period failure on reporting "dispatcher encounter error"
  • [HAWQ-162] - YARN containers are required evenly
  • [HAWQ-163] - Null container set instance is not checked when allocating vseg from resource pool
  • [HAWQ-164] - A typo in the file name of dumped resource queue status.
  • [HAWQ-166] - Orphaned share input scan QE process after OLAP grouping query
  • [HAWQ-169] - pg_dumpall has wrong filespace sql and ignored the tablespace configuration for database.
  • [HAWQ-172] - Add gpoperation.py back for gppkg use
  • [HAWQ-176] - REORGANIZE parameter is useless when change distribute policy from hash to random
  • [HAWQ-180] - Parameter key spelled wrong in hdfs-client.xml
  • [HAWQ-183] - DISCLAIMER should be included in the release.
  • [HAWQ-184] - pass RAT check
  • [HAWQ-185] - clean pxf jars in source code
  • [HAWQ-186] - clean code
  • [HAWQ-187] - Remove useless file in resource manager
  • [HAWQ-188] - Unreleased Resource: stream: AvroResolver.java: Ln 58
  • [HAWQ-189] - XSS: Reflected: WritableResource.java: Ln 149
  • [HAWQ-190] - XSS: Reflected: Invalid PathResource.java: Ln 99, 107
  • [HAWQ-192] - Fail to getContainerStatus and release resource when Yarn HA is enable
  • [HAWQ-193] - Remove libhdfs config and codes from libyarn
  • [HAWQ-195] - Correct the spelling of parameter rpc.client.socekt.linger.timeout
  • [HAWQ-197] - Adding a huge number of GRM containers cause socket connection timed out.
  • [HAWQ-198] - Build-in functions will be executed in resource_negotiator stage which causes it will be executed twice
  • [HAWQ-199] - Add License Header for PXF
  • [HAWQ-201] - Resource uneven or fragment problem may cause resource queue goes into long term pause
  • [HAWQ-202] - lock logic issue when check item existing in metadata cache
  • [HAWQ-204] - long dbname causes core dump
  • [HAWQ-205] - PXF Throws NullPointerException when DELIMITER missing in Hive table definition
  • [HAWQ-209] - Hide detailed log of metadata cache by default.
  • [HAWQ-212] - Fix unittest failure due to introduced enum RESOURCEPROBLEM
  • [HAWQ-213] - Remove deprecated GUC gp_create_table_random_default_distribution
  • [HAWQ-214] - Build-in functions for gp_partition will cause cores.
  • [HAWQ-216] - Built-in functions gp_update_global_sequence_entry has a bug
  • [HAWQ-217] - "invalid memory alloc request size" during "SELECT * FROM t WHERE w ORDER BY o"
  • [HAWQ-218] - Core dump while selecting data from partitioned table with where condition that belongs to no partition
  • [HAWQ-219] - Built-in function gp_persistent_build_all doesn't reshape gp_persistent_relation_node table
  • [HAWQ-220] - Need to add retry for handling RUALive message From RM
  • [HAWQ-224] - make distprep: /contrib/Makefile: error: commands commence before first target
  • [HAWQ-226] - Core dump caused by data locality.
  • [HAWQ-227] - Data locality downgrade by wrong insert host.
  • [HAWQ-228] - Add License Header for gpopt
  • [HAWQ-229] - External table can be altered, which make errors.
  • [HAWQ-230] - fix build failure on centos 7
  • [HAWQ-232] - Crash on INSERT into big partition table will cause a long time recovery
  • [HAWQ-233] - HAWQ failed to initialize the cluster if it is built on centos 7
  • [HAWQ-235] - HAWQ init report error message on centos7
  • [HAWQ-236] - Meta data hit ratio is -0.0 when file len is zero.
  • [HAWQ-237] - failed to acquire resource because of no available resource to run query in multiple stream run query in yarn mode
  • [HAWQ-238] - Sending wrong size of message for returning YARN container
  • [HAWQ-240] - Set Progress as 50% When Return Resource
  • [HAWQ-241] - Cleanup MirroredLock related codes before checkpoint lock holding time optimization.
  • [HAWQ-244] - Cannot cancel the query after the resource context recycled du to timeout error
  • [HAWQ-250] - Fix building and packaging issue after removal of deprecated GUCs
  • [HAWQ-252] - Coredump When RM Reconnect libyarn
  • [HAWQ-253] - Separate pxf-hdfs and pxf-hive packages from pxf-service
  • [HAWQ-254] - Init standby fails if segments in 'slaves' file not inicialized
  • [HAWQ-255] - Checkpoint is blocked by TRANSACTION ABORT for INSERTING INTO a big partition table
  • [HAWQ-260] - Resource manager wrongly calculates number of vseg
  • [HAWQ-261] - No return statement in function setSegResGLOBAvailability()
  • [HAWQ-262] - Wrong string buffer size in snprintf in function callSyncRPCRemote()
  • [HAWQ-263] - "Wrong sizeof argument (SIZEOF_MISMATCH)" issue in MoveOutCGroupForQE found by Coverity
  • [HAWQ-266] - Check if krb_server_keyfile exist in hawq-site.xml
  • [HAWQ-267] - AM heatbeat thread quits, HAWQ AM should reconnect libyarn
  • [HAWQ-268] - HAWQ activate standby fails
  • [HAWQ-270] - make -C contrib distclean is broken
  • [HAWQ-272] - Segment status will not be down after killing postmaster process of segment
  • [HAWQ-273] - Concurrent read committed SELECT return 0 rows for AO table which is ALTERed with REORGANIZE by other transaction
  • [HAWQ-275] - After killing QE of segment, the QE pool is not updated when dispatch
  • [HAWQ-278] - Add min_cost for each query.
  • [HAWQ-280] - Error accessing external table or copying from file with bad rows
  • [HAWQ-285] - not freed connection track instance
  • [HAWQ-286] - Queued resource requests are not adjusted along with the changing of cluster capacity
  • [HAWQ-292] - Remove debugging information in base.py due to introduction of hawqfaultinjector
  • [HAWQ-295] - New metadata flush strategy remove 1 entry every time flush due to flush condition wrong.
  • [HAWQ-296] - TPC-H Query 5 encounters OOM in large HAWQ cluster with None mode
  • [HAWQ-298] - Double check hdfs length longer than metadata logic length.
  • [HAWQ-299] - Extra ";" in udpSignalTimeoutWait
  • [HAWQ-300] - Out of memory during loading large volume data on large cluster with YARN mode
  • [HAWQ-302] - hawq_toolkit.hawq_size_of_database generate warning "get hdfsFileInfo numEntries invalid in gpfs_hdfs_freefileinfo" on large cluster
  • [HAWQ-314] - AO read error due to wrong init of bufferedRead->largeReadLen
  • [HAWQ-316] - Build on Mac report error
  • [HAWQ-318] - Infinite loop when adjusting resource request in one resource queue track
  • [HAWQ-320] - Can't get current user name if $USER is not set
  • [HAWQ-322] - Drop tablespace problem: doesn't check database/relations in some cases
  • [HAWQ-323] - Cannot query when cluster include more than 1 segment
  • [HAWQ-325] - In 3-node cluster HAWQ RM does not allocate resource to queries if two nodes have no resource allocated from YARN
  • [HAWQ-328] - plsql loop three times exit abnormality
  • [HAWQ-329] - Remove statement_mem and deprecated resource queue policy setting
  • [HAWQ-330] - Fail to recognize value of activestats in catalog table pg_resqueue when the value is large
  • [HAWQ-332] - Core dump when rebuilding resource requests in resource queue shadow
  • [HAWQ-334] - installcheck fail due to cannot locate Env.pm during compilation on CentOS 7.2
  • [HAWQ-335] - Cannot query parquet hive table through PXF
  • [HAWQ-341] - PL functions core dump due to query resource is not properly allocated for function/table function/sub-query
  • [HAWQ-343] - Core when setting enable_secure_filesystem to true
  • [HAWQ-344] - When resource queue capacity is shrunk, deadlock detection maybe not triggered
  • [HAWQ-345] - Can not drop a resource queue that is busy
  • [HAWQ-346] - Windown function explain cause dispatch core
  • [HAWQ-347] - Dead Lock Issue when concurrent use catalog and persistent table.
  • [HAWQ-350] - Disable some of installcheck tests due to plpython is not installed by default
  • [HAWQ-353] - Segment number mismatch in RM when kill a segment
  • [HAWQ-355] - order by problem: sorting varchar column with space is not correct.
  • [HAWQ-356] - varchar concat problem: one item is null || result would be NULL.
  • [HAWQ-359] - Update bug reporting location for "./configure --help"
  • [HAWQ-360] - Data loss when alter partition table by add two column in one time.
  • [HAWQ-361] - Data loss when alter partition table by add two column in one time.
  • [HAWQ-362] - Master RM should Recalculate Segment's Allocation Status if Segment's RM is restart.
  • [HAWQ-368] - Core when canceling a query
  • [HAWQ-372] - Single row insert and COPY hang in high concurrent workloads
  • [HAWQ-373] - QE coredump in pg_encoding_mbcliplen when run concrurrent copy in stress
  • [HAWQ-374] - hawq core dump at dispatch_free_result():1494 when run stress test
  • [HAWQ-375] - Fixed hawq core dump at dispatch_free_result():1493 when run stress test
  • [HAWQ-376] - QE core dump at HdfsCloseFile() when run stress test with COPY
  • [HAWQ-377] - Reindex should not be dispatched to segments
  • [HAWQ-380] - Wrongly validate resource pool status when index of allocated resources is not complete
  • [HAWQ-381] - Fatal error when canceling qeury
  • [HAWQ-382] - Adjust combined workload index in resource pool in different situations
  • [HAWQ-383] - Fix memory leak when testing local postmaster health
  • [HAWQ-384] - hawq init fail duo to ssh strict host key checking
  • [HAWQ-385] - hawq failed to initialize due to too many concurrent ssh connection to master or standby
  • [HAWQ-387] - Big NVSEG_LOWER_LIMIT causes resource request hung
  • [HAWQ-389] - libxml2 reader instance is not freed after reading hawq-site.xml
  • [HAWQ-390] - Memory leaks in dispatcher: QD uses 48G memory during ANALYZE lasting 1 ~ 2 days on 100 nodes cluster
  • [HAWQ-391] - bug when failed to acquire resource from resource manager
  • [HAWQ-392] - HAWQ init report ERROR but finished successfully
  • [HAWQ-395] - User can only alter MEMORY_LIMIT_CLUSTER/CORE_LIMIT_CLUSTER if the target queue is a branch queue
  • [HAWQ-396] - When we run 4 con on tpch25g data, coredump in AbortTransaction
  • [HAWQ-402] - Core dumped in random table since sortcl is NULL in tlist.c
  • [HAWQ-403] - Nested decode fails when inner decode return null
  • [HAWQ-405] - Issue with handling pg_temp schema
  • [HAWQ-407] - memory leak when insert into parquet table
  • [HAWQ-409] - dangling pointer is used in AORelRemoveHashEntry
  • [HAWQ-411] - Remove dependency on thrift_devel library if redundant
  • [HAWQ-412] - Allocate query resource for multiple EXECUTIONs in explicit and implicit prepared statement
  • [HAWQ-413] - Create_table_distribution sporadically fails.
  • [HAWQ-414] - Share memory become inconsistent if the number of segment files created more than max_appendonly_segfiles
  • [HAWQ-416] - core dump when cancel delegation token if fatal error happened when setup connection to NAMENODE
  • [HAWQ-417] - QE coredump in MemoryContextFreeImpl when run parquet data type test case
  • [HAWQ-418] - Core dump when COPY data into table without any column
  • [HAWQ-419] - Failed to MemoryAccounting_SaveToLog in resource negotiator.
  • [HAWQ-420] - Memory leaks in DataLocality during ANALYZE lasting 1 ~ 2 days on 100 nodes cluster.
  • [HAWQ-423] - Fix gradle and download plugin version
  • [HAWQ-424] - When HAWQ RM keeps getting 0 GRM containers query resource request is not timed out
  • [HAWQ-425] - In Yarn mode, segment can start without hawq_rm_memory_limit_perseg and hawq_rm_nvcore_limit_perseg configured
  • [HAWQ-426] - Memory leak for query and prepared statement due to introduction of query resource parameters
  • [HAWQ-427] - Wrong results in mdqa+rollup
  • [HAWQ-428] - ORDER/GROUP BY not found in target list error found in olap query
  • [HAWQ-429] - Fix typo in HostnameVolumnInfo and HostDataVolumnInfo
  • [HAWQ-432] - Memory leaks in pg_regress
  • [HAWQ-438] - EagerlyReleased hash table involved in hash join in explain statement introduce core on hawq dbg build.
  • [HAWQ-439] - Master Resource Manager Process Should Exit If Postmaster is Not Alive
  • [HAWQ-440] - Fix bug in pg_regress of commit HAWQ-432.
  • [HAWQ-441] - remove catalog table "gp_id" from code
  • [HAWQ-442] - Wrong results in rollup which contains the same expr targetlist
  • [HAWQ-443] - Add GUC name validation and sync hawq-site.xml for hawq config
  • [HAWQ-444] - The counter for increasing and decreasing memory quota is not correctly subtracted
  • [HAWQ-446] - Master postgres initdb failed(hawq init cluster)
  • [HAWQ-447] - YARN mode resource manager does not count available segment back in to available segment counter
  • [HAWQ-449] - Should not adjust FTS resource capacity in YARN mode
  • [HAWQ-453] - Do not allocate query resource in prepare stage for prepared statement
  • [HAWQ-456] - Defaults file name is hardcoded
  • [HAWQ-457] - Cannot create madlib schema
  • [HAWQ-458] - Wrongly judge validate connection context progress for failing to acquire resource
  • [HAWQ-460] - thrift_devel library should be provided
  • [HAWQ-462] - Querying Hcatalog in HA Secure Environment Fails
  • [HAWQ-463] - Fix the version in the PXF API doc
  • [HAWQ-464] - Update pxf port number in comments
  • [HAWQ-471] - Reindex bug
  • [HAWQ-472] - Add checks to HAWQ standby start up
  • [HAWQ-474] - Core dump in tpcds query
  • [HAWQ-476] - Unexpected internal error in execHHashagg.
  • [HAWQ-477] - Copy table to file do not real execute for lineitem.
  • [HAWQ-478] - Bug when shut down cluster during recovery pass3
  • [HAWQ-480] - HAWQ RM does not calculate correct yarn cluster volume
  • [HAWQ-481] - Optimise analyse virtual segment number for random tables .
  • [HAWQ-483] - Error in debug1 state
  • [HAWQ-484] - Fix bug in gpcopy and pgdump bug.
  • [HAWQ-485] - Wrong results in rollup with plan_append_aggs_with_rewrite
  • [HAWQ-487] - Ensure virtual segment of analyze not greate than GetUtilPartitionNum.
  • [HAWQ-488] - Cannot drop schema madlib and re-deploy MADlib in schema madlib
  • [HAWQ-489] - Missing errno for libhdfs function call
  • [HAWQ-494] - Add checks to standby start/init
  • [HAWQ-495] - Core in cdbexplain_depositStatsToNode
  • [HAWQ-496] - Add unit to output of datalocality info of analyze.
  • [HAWQ-499] - IPv6 address not read properly on OSX version < 10.11
  • [HAWQ-503] - Failed Tmp Dir and GRM Host/rack conflicts in YARN mode
  • [HAWQ-504] - Crash when WALSendServer and standby connection timeout
  • [HAWQ-505] - Fix InputFormatError caused by GUC change.
  • [HAWQ-506] - Fix coverity issue for HAWQ-455
  • [HAWQ-508] - Fix pointer check issue in coverity
  • [HAWQ-509] - Run "select * from pg_stat_activity" returns multiple records for one query
  • [HAWQ-510] - Interconnect error(Could not pare remote listeneraddress '[local]') during running query with generate_series
  • [HAWQ-512] - Query hang due to deadlock in entrydb catalog access
  • [HAWQ-515] - On Mac 10.10.5, run installcheck-good, there are some warnings around tmpnam
  • [HAWQ-516] - gpfdist makefile wrong dependency: install does not depend on make
  • [HAWQ-517] - hawqextract column context does not exist error
  • [HAWQ-518] - hawqextract column context does not exist error
  • [HAWQ-519] - hawqextract column context does not exist error
  • [HAWQ-520] - hawqextract column context does not exist error
  • [HAWQ-521] - external table test failure in installcheck-good with orca disabled
  • [HAWQ-524] - Do not resolve the condition of 'executor->refResult = NULL' in executormgr_bind_executor_task()
  • [HAWQ-525] - query dispatcher heart-beat threads should not close FD if FD is not open ever
  • [HAWQ-526] - Resource queue total resource request is not updated when cluster capacity is changed
  • [HAWQ-527] - Remove dead code of generateResourceRefreshHeartBeat()
  • [HAWQ-528] - Reset gp_connections_per_thread for dispatcher guc range from 1 to 512, 0 marks as invalid.
  • [HAWQ-533] - Cursor failed, if don't allocate resource in prepare.
  • [HAWQ-534] - A query resource quota can be calculated to from 0 to 0
  • [HAWQ-535] - hawqextract column context does not exist error
  • [HAWQ-536] - Regress test 'function' fails when orca is not enabled
  • [HAWQ-540] - Change output.replace-datanode-on-failure to true by default
  • [HAWQ-541] - When YARN containers disappear from container report, the segment available resource maybe negative
  • [HAWQ-542] - Passing empty string to attribute vseg_resource_quota makes default value set instead of reporting error
  • [HAWQ-543] - src backend makefile fails when you try to run unittest's target
  • [HAWQ-544] - Wrong MemoryAccountingPeakBalance assert after calling elog in function AdvanceMemoryAccountingGeneration, calling multiple errors thrown when OOM hit
  • [HAWQ-551] - Add datalocility time to explain analyse result
  • [HAWQ-552] - long latency of dispatching query resource due to wrong processing routine
  • [HAWQ-553] - Analyze ROOTPATITION using too few vsegs.
  • [HAWQ-554] - Create web external table shouldnot omit ON clause.
  • [HAWQ-555] - Remove unused guc for allocating resource for tolerating variance of number of virtual segments
  • [HAWQ-556] - Resource manager uses unadjusted water level guc for allocating GRM resource containers
  • [HAWQ-558] - The query running on random table does not push resource manager to request resource from YARN as maimum possible resource consumption
  • [HAWQ-559] - QD hangs when QE is killed after connected to QD
  • [HAWQ-561] - Wrongly check variance of number of virtual segments
  • [HAWQ-562] - Refactor bucket number of external table.
  • [HAWQ-563] - Restore guc hawq_rm_nvseg_variance_amon_seg_respool_limit and fix its bug when resource manager resets resource
  • [HAWQ-564] - QD hangs when connecting to resource manager
  • [HAWQ-566] - Fix analyze and truncate hung.
  • [HAWQ-568] - After query finished, kill a QE but can still recv() data from this QE socket
  • [HAWQ-569] - Assertion failure because of multiple ExecutorEnd calls
  • [HAWQ-570] - YARN queue report should be a relative percentage value instead of an absolute value
  • [HAWQ-571] - Fix Misspellings in libyarn
  • [HAWQ-573] - Resource leak when cancel copy
  • [HAWQ-574] - Memory Leak in dispatcher
  • [HAWQ-575] - QE core dumped when report "Resume interrupt holdoff count is bad (0) (xact.c:2907)"
  • [HAWQ-577] - Stream PXF metadata response
  • [HAWQ-578] - Avoid validating whether total allocated resource is more than maximum YARN resource queue capacity
  • [HAWQ-580] - Set analyze vseg number for partition and no-partition table separately.
  • [HAWQ-581] - Memory Leak in QD to RM communication context
  • [HAWQ-582] - Handle ApplicationMasterNotRegisteredException in libyarn
  • [HAWQ-585] - Invalid execution statistics received stats node-count mismatch
  • [HAWQ-590] - Remove Dead Codes in resourcepool.c and requesthandler_ddl.c
  • [HAWQ-591] - Explain analyze should not gather QE execution detail when QE runtime error has been caught by QD
  • [HAWQ-592] - QD fails when connects to QE again in executormgr_allocate_any_executor()
  • [HAWQ-593] - Function crashes with PANIC in planning stage related to gp_cte_sharing
  • [HAWQ-598] - Dispatch core in dispatch_free_result is stress test
  • [HAWQ-602] - Remove unused code for utility components in resource manager
  • [HAWQ-603] - Remove unused code for resource manager segment side
  • [HAWQ-604] - Remove Dead Codes in resourcebroker_API.c
  • [HAWQ-605] - Some segment capacity changes are not logged out and when segment goes to up status, the capacity is not adjusted
  • [HAWQ-607] - Copy large table to file, and kill QE process, copy doesn't fail.
  • [HAWQ-608] - libyarn hot refactor
  • [HAWQ-609] - ERROR: Master unable to connect to seg0 DataNode3:0: invalid port number: "0"
  • [HAWQ-611] - Improve performance for query with entry database involved
  • [HAWQ-612] - Improve error message if unavailable segments exceeds GUC
  • [HAWQ-613] - Cannot cancel query when PL UDF has infinity loops
  • [HAWQ-616] - Mininum resource needed by copy random table from a file should be one.
  • [HAWQ-617] - Add a flag to hawq config to allow skipping hosts on which ssh fails and continue with syncing configurations files
  • [HAWQ-620] - Remove useless codes in resourcebroker_NONE.c
  • [HAWQ-621] - TupleDescriptor leak during split_partition
  • [HAWQ-622] - fix libhdfs3 readme
  • [HAWQ-623] - Resource quota request does not follow latest resource quota calculating logic
  • [HAWQ-624] - Copy a large table to output file, multiple QEs are started, but only one QE is assigned actual task.
  • [HAWQ-626] - HAWQ stop segments should check if node alive first
  • [HAWQ-627] - When we run 128 partition in 128 nodes, it core in metadata cache
  • [HAWQ-628] - \l+ returns hcatalog not supported error
  • [HAWQ-629] - Insert into table select generate_series free resource too early.
  • [HAWQ-631] - Wrongly report error message when QD fails to connect to resource manager
  • [HAWQ-632] - HAWQ remove offline standby fails
  • [HAWQ-633] - Do not error out when cleaning up workfiles during AbortTransaction
  • [HAWQ-635] - QE process does not exit in libhdfs
  • [HAWQ-637] - RM process is error if property is missing in hawq-site.xml
  • [HAWQ-638] - gpload bug using pip installed PyGreSQL
  • [HAWQ-640] - Wrongly timing the reqeusts for YARN containers causes resource manager not fully acquire resource
  • [HAWQ-642] - Appendonlywriter.c assign useless segfilestatus for invalid segno
  • [HAWQ-644] - Failure in security ha environment with certain writable external tables
  • [HAWQ-647] - configure compilation flag bug
  • [HAWQ-650] - Redundent idle qe for copy table to file.
  • [HAWQ-652] - Slicemap in dispatch is mismatched when run multiple query in one session.
  • [HAWQ-653] - libyarn code coverage build does not work
  • [HAWQ-655] - Allocate resource for PerformSharedStorageOpTasks bug.
  • [HAWQ-656] - gppkg can't remove installed module
  • [HAWQ-658] - Kill/Stop Hadoop YARN RM, Segments' Status is Still UP
  • [HAWQ-660] - Change segment resource manager log level when failing in updating memory quota
  • [HAWQ-662] - Fix memory leak when mppExecutorCleanup throws error.
  • [HAWQ-663] - Wrongly count total allocated resource in resource queue manager
  • [HAWQ-664] - The return value of registering fd is not checked for asynchronous connection
  • [HAWQ-666] - Restore to correct memory context after free query resource in ExecutorEnd
  • [HAWQ-667] - QD resource heart-beat sleeps too long time.
  • [HAWQ-669] - Interconnect guc gp_interconnect_transmit_timeout type error
  • [HAWQ-671] - Validation raises error when adding accepted containers into a down node and fix function typo
  • [HAWQ-672] - Add python module pygresql back into hawq workspace
  • [HAWQ-673] - Unify out put of explain analyze.
  • [HAWQ-674] - Wrongly validate an incomplete resource pool status when GRM is not available
  • [HAWQ-675] - pg_hba configuration not updated while hawq init standby
  • [HAWQ-676] - Apply fix for common codes to libyarn
  • [HAWQ-677] - During activate standby ignore-bad-hosts flag is not passed to the command
  • [HAWQ-678] - Resource manager should close connection with QD when QD is cancelled and try to return resource to clean up all registered resource context
  • [HAWQ-679] - During remove standby host if a segment host is down remove standby host fails
  • [HAWQ-682] - hawq init master fails to syncup hawq-site xml if there is a segment host down
  • [HAWQ-683] - Generate javadoc for PXF project for HAWQ2.0
  • [HAWQ-684] - Wrongly process container set when failed to increase resource quota due to failing to connect to segment
  • [HAWQ-685] - Pass wrong pointer to transform connection track instance status when canceling a waiting query resource request
  • [HAWQ-687] - The segment resource increase pending counter is not adjusted correctly when failing to increase resource quota
  • [HAWQ-688] - Fix infinite loop when hawq fail to register itself to YARN
  • [HAWQ-689] - Create table with (oids) in PBE should check whether arg of oids is NULL.
  • [HAWQ-705] - \d combines hive tables with same prefix but different suffix (numbers) only
  • [HAWQ-711] - Integrate libhdfs3 and libyarn makefile into hawq
  • [HAWQ-712] - Add GUC gp_vmem_idle_resource_timeout back to do timeout for gang release in idle session
  • [HAWQ-713] - lc_numeric guc doesn't behave as expected after some time
  • [HAWQ-715] - hawq_rm_yarn_address doesn't take effect if libyarn is in HA mode
  • [HAWQ-717] - HAWQ hawq ssh-exkeys command should consider the string exactly as its given at the command line prompt
  • [HAWQ-726] - configure and configure.in are not consistent.
  • [HAWQ-727] - Fix featuretest makefile error.
  • [HAWQ-729] - Restore make installcheck in makefile.
  • [HAWQ-737] - Last action time should be initialized as created time instead of 0
  • [HAWQ-738] - Allocate query resource twice in function call through jdbc
  • [HAWQ-744] - Add plperl code
  • [HAWQ-745] - Fix header miss in test library on Redhat.
  • [HAWQ-746] - ssh_utils.py:addHostNameAlternatives() should not use short hostname
  • [HAWQ-751] - Add plr, pgcrypto, gporca into Apache HAWQ
  • [HAWQ-755] - query fail when resource management use yarn
  • [HAWQ-756] - Fail to compile with configuration "--enable-snmp --disable-email"
  • [HAWQ-759] - Query cannot be terminated by pg_cancel_backend or pg_terminate_backend
  • [HAWQ-767] - make hawq code work with upstream libesmtp
  • [HAWQ-770] - .gitignore depends/thirdparty/thrift/hawqbuild/Makefile.global
  • [HAWQ-772] - Hotfix for HAWQ-760
  • [HAWQ-773] - Need to install gpos and gp-xerces also when gporca is enabled.
  • [HAWQ-777] - Add back gpfdist
  • [HAWQ-780] - Remove quicklz compression related code but keep related meta data in short term.
  • [HAWQ-785] - Failure running `make -j8 all`
  • [HAWQ-792] - Orca on causes different error message when insert null value to a not null col
  • [HAWQ-793] - Temporarily remove the snappy info in metadata but keep the snappy support for row oriented storage.
  • [HAWQ-800] - Less tuple is inserted due to data locality information is not refreshed and dispatched in prepared statement
  • [HAWQ-812] - Activate standby master failed after create a new database
  • [HAWQ-813] - Activate standby master failed after create a new database
  • [HAWQ-815] - KeyError while compiling PLPython function due to deletion of non-existent record from Python global dict
  • [HAWQ-819] - Wrong timestamp file path for Makefile target pg_prepare
  • [HAWQ-821] - Inner function does not exist while invoking from outer function on mac osx
  • [HAWQ-824] - Move orca submodules to depends/
  • [HAWQ-825] - HAWQ component reload shows misleading message on stdout
  • [HAWQ-826] - Compile plr and install plr with different prefix when building a plr rpm.
  • [HAWQ-833] - Lack of pgcrypto.sql for the pgcrypto support
  • [HAWQ-835] - Cannot retrieve tuple from temp table created in function
  • [HAWQ-839] - Libyarn coredump when failover to standby RM
  • [HAWQ-841] - Running "make -j8 unittest-check" under src/backend will throw "File exists" error
  • [HAWQ-842] - Failed to acquire resource from resource manager
  • [HAWQ-847] - Build PLR error on Mac
  • [HAWQ-848] - Writable external table: gpfdist report "Failed initialization (url.c:1671)"
  • [HAWQ-853] - Master standby should avoid incomplete split operation
  • [HAWQ-854] - Remove directories from %files in plr.spec
  • [HAWQ-859] - Fix the naming of output columns of append node
  • [HAWQ-861] - Wrong logerror position in insert into hashtable select * from gpfdist external table.
  • [HAWQ-862] - Make user defined function get_ao_distribution work.
  • [HAWQ-865] - Rebase upstream pgcrypto to a newer commit which includes a critical DES crypt() bug fix
  • [HAWQ-867] - Replace the git-submobule mechanism with git-clone
  • [HAWQ-868] - README.md needs to update
  • [HAWQ-871] - HAWQ CHECK looking for wrong YARN HA parameters
  • [HAWQ-872] - HAWQ check fails if '--kerberos' not given when kerberos is actually enabled
  • [HAWQ-876] - Add the support for initFile option of gpdiff.pl in hawq googletest framework.
  • [HAWQ-877] - Need to stop building when building error occurs in gpfdist directory
  • [HAWQ-880] - Output of 'hawq stop --reload' is not correct
  • [HAWQ-882] - insert failed when compresstype=gzip
  • [HAWQ-885] - Should fail earlier at creating for a table with quicklz compression.
  • [HAWQ-887] - Query should be continued when bucket number of result hash relation is bigger than the on number of command external table.
  • [HAWQ-888] - Remove some dummy files which were used to keep related "empty" directories.
  • [HAWQ-889] - Tweak the error output for quicklz table creation.
  • [HAWQ-891] - Refine libyarn source codes
  • [HAWQ-892] - Need "incubating" in Apache HAWQ release name
  • [HAWQ-893] - Incorrect array indexing in load_expected_statuses() in pg_regress.c
  • [HAWQ-901] - hawq init failed: hawqstandbywatch.py:test5:gpadmin-[WARNING]:-syncmaster not running
  • [HAWQ-903] - Fix pxffilters_test failure and HBase filter failure
  • [HAWQ-915] - RAT is failing on the latest master
  • [HAWQ-933] - Build warnings
  • [HAWQ-937] - Clean up "Pivotal Extension Framework" in ReadMe.md
  • [HAWQ-976] - Incompatible BSD 4-clause licenses
  • [HAWQ-993] - Symbolic link (contrib/pgcrypto) pointing to non-existent location
  • [HAWQ-1007] - Add the pgcrypto code into hawq
  • [HAWQ-1259] - Remove gprecoverseg from documentation
  • [HAWQ-1390] - HAWQ build on s390x
  • [HAWQ-1391] - s390x support for HWCRC32c
  • [HAWQ-1412] - Inconsistent json file for catalog of hawq 2.0

New Feature

  • [HAWQ-25] - Add resource queue new ddl statement implementation
  • [HAWQ-38] - Support HA for libyarn
  • [HAWQ-44] - Advanced statistics for PXF tables
  • [HAWQ-47] - Make user able to set statement level resource usage.
  • [HAWQ-51] - Support Kerberos for libyarn
  • [HAWQ-76] - add guc to determine whether to convert hash table to random with enforcement
  • [HAWQ-88] - set default value of guc hash_to_random_flag to be ENFORCE_KEEP_HASH
  • [HAWQ-144] - Build HAWQ on MacOS
  • [HAWQ-178] - Add JSON plugin support in code base
  • [HAWQ-194] - Test facility for pausing global resource manager container lifecycle in resource pool
  • [HAWQ-276] - Bump Orca and GPOS version in HAWQ to get MDCache / CCache eviction
  • [HAWQ-281] - User can alter any resource queue no matter whether its busy or at least one its descendent queue is busy
  • [HAWQ-288] - Support hawqfaultinjector as an internal utility for fault injection purpose - client side
  • [HAWQ-289] - Support hawqfaultinjector as an internal utility for fault injection purpose - server side
  • [HAWQ-290] - Support hawqfaultinjector as an internal utility for fault injection purpose - modify fault type and name
  • [HAWQ-293] - Enable writable external table to support on number syntax.
  • [HAWQ-305] - Support PostGIS
  • [HAWQ-306] - Direct hdfs data file drop & load to native HAWQ tables
  • [HAWQ-307] - Ubuntu Support
  • [HAWQ-317] - PXF supports Secure Isilon Cluster
  • [HAWQ-351] - Add movefilespace option to 'hawq filespace'
  • [HAWQ-369] - Hcatalog as reserved name
  • [HAWQ-393] - HAWQ access of hcatalog metadata using \d
  • [HAWQ-431] - Add option to 'hawq init' to specify standby host name
  • [HAWQ-523] - Dead code in executormgr_bind_executor_task()
  • [HAWQ-538] - Add fault injection for dispatcher
  • [HAWQ-550] - Automatically clean up records in gp_configuration_history longer than a GUC value
  • [HAWQ-560] - Set 'output.replace-datanode-on-failure' depend on datanodes number
  • [HAWQ-597] - Add Travis CI support under OSX
  • [HAWQ-665] - Dump memory usage information during runaway query termination
  • [HAWQ-703] - Serialize HCatalog Complex Types to plain text (as Hive profile)
  • [HAWQ-709] - Include googletest for unittest/featuretest for HAWQ
  • [HAWQ-721] - New feature test skeleton
  • [HAWQ-722] - Add Doc around how to use HAWQ via JDBC/ODBC/libpq et al.
  • [HAWQ-731] - Implement Data Generator
  • [HAWQ-774] - Add snappy compression support to row oriented storage
  • [HAWQ-789] - Explicitly initialize GPOPT and its dependencies
  • [HAWQ-810] - Add string format for feature test library
  • [HAWQ-817] - Remove deprecated test cases from checkinstall-good
  • [HAWQ-856] - Explicitly initialize GPOPT and its dependencies.

Improvement

  • [HAWQ-5] - Update HAWQ to support latest hadoop version and ecosystem
  • [HAWQ-13] - Close gap between internal stash and public git repo
  • [HAWQ-28] - Fix JavaDoc warnings and errors introduced by Java8
  • [HAWQ-40] - Do standby init from standby node instead of master node
  • [HAWQ-46] - Change data type for Metadata Versioning versions from int64 to uint64
  • [HAWQ-49] - Remove legacy madlib schema in HAWQ
  • [HAWQ-55] - System admin cannot set optimizer_release_mdcache (OFF) when gp_metadata_versioning is (OFF)
  • [HAWQ-72] - Introduce ADDON_DIR macro in configure/makefile to allow customization
  • [HAWQ-80] - Support dynamic relation distribution type in gpopt
  • [HAWQ-81] - Make resource manager to reject resource request when there are too many segments down
  • [HAWQ-82] - Allocate resource for EXPLAIN statements
  • [HAWQ-97] - Improve performance of Metadata Versioning by eliminating redundant versioning events
  • [HAWQ-111] - Rename guc variable names
  • [HAWQ-114] - Set resource queue default parallel limit to 20
  • [HAWQ-119] - Log gpcheckhdfs output and Suppress unwanted messages
  • [HAWQ-120] - Add grace error message after namespace change
  • [HAWQ-121] - Remove legacy command line tools
  • [HAWQ-123] - Add 'Partition by' description in psql
  • [HAWQ-132] - Enrich log information when queuing resource request is timed-out.
  • [HAWQ-137] - Update hdfs-client.xml and yarn-client.xml
  • [HAWQ-140] - Add more information in HAWQ build instructions file
  • [HAWQ-143] - Add informations in Apache-HAWQ README.md
  • [HAWQ-152] - Pass validation test in resource manager for unexactly divisible memory core ratio
  • [HAWQ-153] - Remove expired code using sigar library in resource manager
  • [HAWQ-154] - Update BUILD_INSTRUCTIONS file for dependencies install method and ambiguous word.
  • [HAWQ-157] - Add refactored implementation of allocating vsegs in resource pool
  • [HAWQ-158] - Remove legacy command line tools which not in use
  • [HAWQ-161] - Port GPDB planner fixes to HAWQ
  • [HAWQ-165] - PXF loggers should all be private static final
  • [HAWQ-168] - Reduce HAWQ start/stop time when having hundreds nodes
  • [HAWQ-170] - Rename GUC gp_vmem_protect_limit to hawq_re_memory_overcommit_max to reflect latest memory enforcement implementation
  • [HAWQ-177] - finalize resource queue ddl error messages
  • [HAWQ-179] - Provide getVersion Interface in PXF REST API
  • [HAWQ-191] - Remove Analyzer from PXF
  • [HAWQ-196] - Refine GUCs in hawq-site.xml and template-hawq-site.xml
  • [HAWQ-200] - Investigate AOSegfileStatus leak
  • [HAWQ-203] - Add a guc for debug metadata, datalocality time stat.
  • [HAWQ-210] - Improve data locality by calculating the insert host.
  • [HAWQ-211] - Refine logging and error messages for resource enforcement
  • [HAWQ-222] - Improve Metadata Cache performance when cache hit
  • [HAWQ-223] - Improve AOGetSegFileDataLocation performane
  • [HAWQ-234] - Improve HAWQ resource manager resource allocation algorithm and RPC framework
  • [HAWQ-239] - Reduce the call times of GpPolicy to improve performance on scan empty table.
  • [HAWQ-242] - Remove deprecated GUCs related to resource management
  • [HAWQ-243] - Remove deprecated GUCs related to planner and optimizer
  • [HAWQ-245] - Remove deprecated GUCs related to Hadoop
  • [HAWQ-247] - Remove deprecated GUCs related to FTS
  • [HAWQ-248] - Remove deprecated GUCs related to backoff/query prioritization
  • [HAWQ-249] - Remove deprecated misc GUCs
  • [HAWQ-257] - PXF to honor user defined port in location URL
  • [HAWQ-259] - YARN mode resource manager gets YARN cluster report once before processing query resource request
  • [HAWQ-265] - Change metadata share memory flush strategy to prevent out of share memory problem when create too many hdfs_file.
  • [HAWQ-269] - Change net_disk_ratio from int to double type.
  • [HAWQ-274] - Add disk check for JBOD temporary directory in segment FTS
  • [HAWQ-279] - Add 2 guc in template-hawq-site
  • [HAWQ-283] - Remove the content column from the catalog pg_aoseg.pg_aoseg_xxxxx
  • [HAWQ-284] - Add a udf for new metadata flush strategy testing.
  • [HAWQ-287] - Control whether to pass data locality prefer list to resource manager.
  • [HAWQ-291] - Make resource queue default 256mb as virtual segment resource quota
  • [HAWQ-294] - Rename prefix of logs in pg_log from gpdb-xxx to hawq-xxx
  • [HAWQ-297] - Double check hdfs and split length equals to metadata logic length.
  • [HAWQ-301] - Use combined value to measure segment workload to improve the resource allocation strategy
  • [HAWQ-313] - Fix dereference pointer before null check
  • [HAWQ-324] - Add minirepro to let user dump minimal set of DDL and statistics for a given query
  • [HAWQ-336] - Set default net_disk_ratio to 1.0.
  • [HAWQ-339] - Validate standby host name while read hawq-site.xml
  • [HAWQ-352] - Add Centos/Redhat 7 build instructions to BUILD_INSTRUCTIONS.md file
  • [HAWQ-357] - Track how many times a segment can not get expected containers from global resource manager
  • [HAWQ-363] - Add guc variables for controlling FTS heartbeat interval and timeout
  • [HAWQ-364] - Make resource manager dynamically adjust minimum YARN container count in each segment
  • [HAWQ-365] - Improve DEBUG level log for counting resource queue active connections
  • [HAWQ-366] - Analyse data locality ratio for TPCH query in SFO.
  • [HAWQ-367] - introduce seg_max_connections guc to set segment max_connections setting
  • [HAWQ-371] - Change seg_max_statements default value to 1280
  • [HAWQ-378] - Move command 'gpcheckperf' to 'hawq checkperf'
  • [HAWQ-379] - Remove deprecated GUC gp_fault_action
  • [HAWQ-386] - Remove deprecated catalog table gp_fault_strategy
  • [HAWQ-397] - Remove useless guc: debug_fix_vseg_num.
  • [HAWQ-399] - Enable Clang/LLVM support for compiling SSE42 bits in crc32c.c
  • [HAWQ-400] - Support expected exit codes for regression tests
  • [HAWQ-404] - Add sort during INSERT of append only row oriented partition tables
  • [HAWQ-410] - Incorporate libhdfs3 into HAWQ source tree as direct source dep.
  • [HAWQ-415] - Include pygresql source code in code base
  • [HAWQ-430] - HAWQ resource manager prunes those registered segments having different memory to core ratio
  • [HAWQ-445] - Support large strings (up to a GB) in text_to_array()
  • [HAWQ-448] - hawq_global_rm_type is case sensitive
  • [HAWQ-451] - Remove dead code in utilities of resource manager
  • [HAWQ-454] - 'hawq stop' should return true if master/standby already stopped
  • [HAWQ-455] - Disable creating partition tables with non uniform bucket schema
  • [HAWQ-467] - Add checks to HAWQ master start/stop
  • [HAWQ-468] - HAWQ remove standby should keep the empty data folder
  • [HAWQ-473] - Implement adding a entry into gp_configuration_history when segment' status is changed
  • [HAWQ-475] - Add build_type gcov for code coverage.
  • [HAWQ-479] - Upgrade JSON-C dependency from 0.6 to 0.12
  • [HAWQ-486] - gpcheck can't find namenode with Ambari install PHD
  • [HAWQ-490] - Add version and compatibility detection for HAWQ + GPORCA
  • [HAWQ-492] - PXF Query Push-Down Feature enhancement to enable further push down to Datasource
  • [HAWQ-493] - Limit some commands can only be run on master/standby
  • [HAWQ-497] - Refactor resource related GUC.
  • [HAWQ-498] - Update property value in gpcheck.cnf
  • [HAWQ-502] - Support --enable-orca in autoconf
  • [HAWQ-507] - Remove dead code for cdbpersistentrelfile.c
  • [HAWQ-511] - Improve HAWQ command line tools running speed and logs
  • [HAWQ-514] - Update and add to PXF Rpm 'Summary' and 'Description' metadata
  • [HAWQ-522] - Add socket connection pool feature to improve resource negotiation performance
  • [HAWQ-529] - Allocate resource for udf in resource negotiator.
  • [HAWQ-530] - Explain analyze should include allocate resource/ data locality time.
  • [HAWQ-531] - Change GUC default value.
  • [HAWQ-532] - Optimise vseg number for copy to statement.
  • [HAWQ-545] - Update PXF RPMs with appropriate description and summary
  • [HAWQ-547] - When HAWQ RM does not have valid YARN resource queue status received the cluster report is not accepted
  • [HAWQ-549] - Directory check ignore empty 'pg_log' folder while standby init
  • [HAWQ-557] - Set connection pool connections always NODELAY and KEEPALIVE
  • [HAWQ-567] - Add HAWQ commands option conflict checks
  • [HAWQ-576] - Remove useless code for identity.c
  • [HAWQ-579] - Remove get_tmpdir_from_rm related code
  • [HAWQ-584] - Choosing cluster to core ratio to a multiple of 1GB
  • [HAWQ-586] - HAWQ stop do not check connections while use immediate mode
  • [HAWQ-587] - Set GUC 'default_hash_table_bucket_number' dynamically
  • [HAWQ-588] - Improve performance for truncate with small table
  • [HAWQ-589] - Clean dead code in resourcemanager.c
  • [HAWQ-594] - remove dead code in resqueuemanager.c and requesthandler.c
  • [HAWQ-595] - set guc hawq_rm_nslice_perseg_limit default value to 5000
  • [HAWQ-596] - change gucs hawq_rm_tolerate_nseg_limit and hawq_rm_rejectrequest_nseg_limit to a percentage value
  • [HAWQ-600] - Remove dead code in resource manager communication component
  • [HAWQ-601] - Clear external table code.
  • [HAWQ-610] - Remove resource manager temporary directory facility that is not used any more
  • [HAWQ-618] - Import libhdfs3 for internal management
  • [HAWQ-630] - Change QD lease heart-beat interval to 10s and timeout to 180s
  • [HAWQ-634] - Let resource manager cancel waiting and return allocated resource when unregister connection rpc is called
  • [HAWQ-636] - hawq check should read configurations in /etc/security/limits.d
  • [HAWQ-641] - Add log for changing host capacity that registers before fixing cluster memory to core ratio
  • [HAWQ-645] - shrink LOG level log output from resource manager
  • [HAWQ-646] - Bump up optimizer version number
  • [HAWQ-648] - Remove hawqsite.c unused code
  • [HAWQ-649] - Remove resourcepool.c unused code
  • [HAWQ-651] - Enhance update and append for ugly format xml file
  • [HAWQ-654] - Reform function getQueueTrackByQueueName() and getQueueTrackByQueueOID() to facilitate test
  • [HAWQ-657] - Remove duplicate validation logic in requesthandler_ddl.c
  • [HAWQ-661] - Remove legacy gpextract command
  • [HAWQ-668] - hawq check should able to check yarn settings
  • [HAWQ-681] - Remove Hcatalog GUC
  • [HAWQ-707] - Remove google test dependency from libhdfs3 and libyarn folder
  • [HAWQ-708] - Consume latest GPORCA features up to version 1.630
  • [HAWQ-716] - Support code coverage build and report in hawq Makefile
  • [HAWQ-718] - wiki of "Build and Install" for CentOS 7.x has some little issues.
  • [HAWQ-723] - Common test library refine
  • [HAWQ-734] - refactor feature test build
  • [HAWQ-735] - Import thrift-0.9.1
  • [HAWQ-749] - segment registration should consider same ip across nodes
  • [HAWQ-758] - Refactor insert checkinstall cases using new test framework
  • [HAWQ-763] - Remove traceflag that allows ORCA to run in multiple threads
  • [HAWQ-764] - Remove CTranslatorPlStmtToDXL Deadcode
  • [HAWQ-765] - Remove CTranslatorDXLToQuery Deadcode
  • [HAWQ-768] - Refactor c++ code for src/test/feature/lib
  • [HAWQ-775] - Provide a seperate PLR package.
  • [HAWQ-781] - Move src/postgres to depends/thirdparty/postgres
  • [HAWQ-790] - Remove CTranslatorPlStmtToDXL Deadcode
  • [HAWQ-811] - Redefine execute interface in sql_util.h of feature test library to support returning error message.
  • [HAWQ-816] - Redefine semantics of execute interface of feature test library
  • [HAWQ-855] - Make resource manager ask for cluster report quickly when YARN is detected down
  • [HAWQ-857] - Remove dead code in the Algebrizer
  • [HAWQ-873] - Improve checking time for travis CI
  • [HAWQ-1248] - Merge Dockerfiles for HAWQ Dev into HAWQ code base
  • [HAWQ-1250] - Create HAWQ-specific html docs

Test

  • [HAWQ-11] - two test cases fail due to plpython issue
  • [HAWQ-79] - Re-enable unit test for shared input scan
  • [HAWQ-85] - Re-enable unit test for executor including Hash Agg, Workfile, and Subplan
  • [HAWQ-86] - Re-enable unit test for pxf and external storage
  • [HAWQ-106] - Enable the exttab1 installcheck-good case
  • [HAWQ-225] - Add aggregate with grouping sets test to hawq
  • [HAWQ-408] - Modify and Pass Libyarn Unit Test Cases
  • [HAWQ-724] - Add hawq config test common library.
  • [HAWQ-725] - Refactor feature test framework.
  • [HAWQ-733] - Command utility takes about 75 sec to process result
  • [HAWQ-739] - Add doxygen template to generate test common lib document.
  • [HAWQ-741] - Set system login user as default when PGUSER is not set.
  • [HAWQ-750] - Test prepare and sequence using new test library
  • [HAWQ-754] - Open googletest code inside hawq for test usage
  • [HAWQ-757] - Hotfix feature-test building bug.
  • [HAWQ-766] - Add feature test for partition using new feature test framework
  • [HAWQ-769] - Add feature test for function, function_extensions and set_functions of user-defined function using new feature test framework
  • [HAWQ-776] - Refactor alter table checkinstall cases using new test framework
  • [HAWQ-791] - remove parquet related test from installcheck-good
  • [HAWQ-801] - Refactor create_aggregate checkinstall cases using new test framework.
  • [HAWQ-803] - Enable filter in code coverage.
  • [HAWQ-805] - Refactor feature test for exttab1 with new framework
  • [HAWQ-809] - Change libhdfs3 function test port to hadoop default port
  • [HAWQ-814] - Enhance user-defined function by migrating create_function_1 of UDF from installcheck to new feature test framework
  • [HAWQ-820] - Correct expected output for query in basic udf suite in new feature test framework
  • [HAWQ-822] - Add string replacement utility in feature test framework to support convert from source to sql and ans files
  • [HAWQ-827] - Refactor aggregate_with_groupingsets checkinstall cases using new test framework.

Task

  • [HAWQ-1] - Remove .p4ignore files from source tree
  • [HAWQ-45] - Package namespace sanitization for PXF: com.pivotal -> org.apache.hawq
  • [HAWQ-57] - Apache License v2.0 File, Notice File Added to Top Level Directory and License Header added to each source file
  • [HAWQ-63] - build and install gpnetbenchServer and gpnetbenchClient
  • [HAWQ-66] - check libxml2 in configure
  • [HAWQ-84] - enable building hawq-hadoop component
  • [HAWQ-96] - Support HBase 1.1.2
  • [HAWQ-100] - Code Cleanup: gpmapreduce
  • [HAWQ-102] - Remove "GPHD" in error message
  • [HAWQ-103] - Separate pxf rpm package logic from code base
  • [HAWQ-105] - Open exttab1 test case in installcheck-good
  • [HAWQ-126] - Switch PXF to use Semantic 3 digit versioning
  • [HAWQ-146] - Re-enable non-uniform partitioning DDLs
  • [HAWQ-148] - Bump up optimizer and gpos version
  • [HAWQ-149] - Add orafce, gp_cancel_query, pgbench and extprotocol to HAWQ
  • [HAWQ-151] - Investigate if Apache HAWQ is vulnerable to Java remote code execution vulnerability
  • [HAWQ-167] - Update HAWQ VERSION to "HAWQ 2.0.0.0-beta"
  • [HAWQ-173] - Set parameter dfs.client.use.legacy.blockreader.local to false in hdfs-client.xml
  • [HAWQ-174] - Set value of dfs.domain.socket.path as /var/lib/hadoop-hdfs/dn_socket
  • [HAWQ-207] - Enable "make RAT" for build process
  • [HAWQ-221] - Cleanup unused files related to optimizer
  • [HAWQ-246] - Refactor and cleanup of memory management of the new optimizer
  • [HAWQ-251] - Add gpfaultinjector back for hawq2.0 using
  • [HAWQ-258] - Investigate whether gp_fastsequence is still needed
  • [HAWQ-264] - Fix Coverity issues
  • [HAWQ-271] - Remove external python modules
  • [HAWQ-327] - Clover License Violation
  • [HAWQ-333] - Remove code for Metadata Versioning
  • [HAWQ-338] - Update build instruction with centos7 steps
  • [HAWQ-342] - Update ASF copyright to 2016 in NOTICE file
  • [HAWQ-349] - Add public KEYS to Apache HAWQ repo.
  • [HAWQ-354] - 2 files missing Apache License Header
  • [HAWQ-388] - Remove unnecessary entries in NOTICE file
  • [HAWQ-394] - Remove pgcrypto from code base
  • [HAWQ-452] - Remove workfile views from hawq_toolkit
  • [HAWQ-461] - README for gpfdist_transform demo contains outdated commands, information
  • [HAWQ-619] - Change 'gpextract' to 'hawqextract' for InputFormat unit test
  • [HAWQ-680] - Remove 'beta' string from Apache HAWQ VERSION
  • [HAWQ-787] - Remove QP Deadcode
  • [HAWQ-863] - Add python module 'pycrypto'
  • [HAWQ-878] - Add googletest cases for the ao snappy compression support.
  • [HAWQ-895] - Investigate migration to 3-digit Semantic versioning
  • [HAWQ-926] - Remove pycrypto from source code
  • [HAWQ-952] - Clean up COPYRIGHT file and review NOTICE File
  • [HAWQ-957] - NOTICE file clean up
  • [HAWQ-958] - LICENSE file missing checklist
  • [HAWQ-959] - Clean up Binary files based on RAT reports
  • [HAWQ-960] - Remove the file BUILD_INSTRUCTIONS.md and move the content in it to README.md

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.