Release Notes - ASF JIRA

Release Notes - Apache Apex Malhar - Version 3.3.0 - HTML format

Configure Release Notes

Sub-task

[APEXMALHAR-1877] - Move org.apache.hadoop.io.file.tfile from contrib to library in Malhar
[APEXMALHAR-1901] - Test- DTFileTest creates test folder under lib directory
[APEXMALHAR-1902] - Rename IdempotentStorage Manager
[APEXMALHAR-1910] - Fix existing checkstyle violations in BlockReader and FileSplitter
[APEXMALHAR-1912] - Fix existing check style violations in FileOutput, JMSInput, FTPInput, JDBC classes
[APEXMALHAR-1916] - Add FileAccess API and its DTFileImplementation
[APEXMALHAR-1931] - Augment FileAccess API
[APEXMALHAR-1941] - Add a default Slice comparator to Malhar/util
[APEXMALHAR-1943] - Add Aggregator to Malhar and make it top level interface
[APEXMALHAR-1944] - Add DimensionsConversionContext to Malhar and make it top class
[APEXMALHAR-1945] - Upgrade the version of japicmp to 0.6.2

Bug

[APEXMALHAR-1880] - Incorrect documentation for maxLength property on AbstractFileOutputOperator
[APEXMALHAR-1887] - shutdown field in WebSocketInputOperator should be volatile
[APEXMALHAR-1894] - Add an Input Port With An isConnected Method
[APEXMALHAR-1922] - [FileStreamContext] Set filterStream variable to transient
[APEXMALHAR-1925] - The kafka offset manager may not store the offset of processed data in all scenarios
[APEXMALHAR-1928] - Update checkpointed offsettrack in operator thread instead of consumer thread
[APEXMALHAR-1929] - japicmp plugin fails for malhar samples
[APEXMALHAR-1934] - When offset is unavailable kafka operator stops reading data
[APEXMALHAR-1947] - Ability to restrict number of messages per window by size for kafka input operator
[APEXMALHAR-1949] - JDBC Input Operator unnecessarily waits two times when the result is empty
[APEXMALHAR-1960] - Test failure KafkaInputOperatorTest.testRecoveryAndIdempotency

New Feature

[APEXMALHAR-1812] - Support Anti Join
[APEXMALHAR-1813] - Support Semi Join
[APEXMALHAR-1904] - New Kafka input operator using 0.9.0 consumer APIs

Improvement

[APEXMALHAR-1895] - Refactor Snapshot Server
[APEXMALHAR-1896] - Add Utility Functions For Working With Schema Tags
[APEXMALHAR-1906] - Snapshot Server support tags
[APEXMALHAR-1908] - Add Deserialization Function That Deserializes keys with multiple values
[APEXMALHAR-1913] - [FileSplitter] Need access to modifiedTime of ScannedFileInfo class
[APEXMALHAR-1918] - [FileSplitter] Need stopScanning method in Scanner
[APEXMALHAR-1940] - Create Operator Utility Class Which Converts Time To Windows
[APEXMALHAR-1958] - Provide access to doneTuple field in AbstractReconciler for derived classes

Task

[APEXMALHAR-1892] - Fix missing javadoc
[APEXMALHAR-1905] - Test the old kafka input operator is compatible with 0.9.0 broker
[APEXMALHAR-1950] - Identify and mark Operators and Components as @Evolving
[APEXMALHAR-1956] - Concrete generic Implementation of Kafka Output Operator with auto metrics and batch processing
[APEXMALHAR-1964] - [Checkstyle] Reduce the severity of line length check

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.

Release Notes - Apache Apex Malhar - Version 3.3.0
    
<h2>        Sub-task
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1877'>APEXMALHAR-1877</a>] -         Move org.apache.hadoop.io.file.tfile from contrib to library in Malhar
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1901'>APEXMALHAR-1901</a>] -         Test- DTFileTest creates test folder under lib directory
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1902'>APEXMALHAR-1902</a>] -         Rename IdempotentStorage Manager
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1910'>APEXMALHAR-1910</a>] -         Fix existing checkstyle violations in BlockReader and FileSplitter
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1912'>APEXMALHAR-1912</a>] -         Fix existing check style violations in FileOutput, JMSInput, FTPInput, JDBC classes
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1916'>APEXMALHAR-1916</a>] -         Add FileAccess API and its DTFileImplementation
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1931'>APEXMALHAR-1931</a>] -         Augment FileAccess API
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1941'>APEXMALHAR-1941</a>] -         Add a default Slice comparator to Malhar/util
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1943'>APEXMALHAR-1943</a>] -         Add Aggregator to Malhar and make it top level interface
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1944'>APEXMALHAR-1944</a>] -         Add DimensionsConversionContext to Malhar and make it top class
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1945'>APEXMALHAR-1945</a>] -         Upgrade the version of japicmp to 0.6.2
</li>
</ul>
            
<h2>        Bug
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1880'>APEXMALHAR-1880</a>] -         Incorrect documentation for maxLength property on AbstractFileOutputOperator
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1887'>APEXMALHAR-1887</a>] -         shutdown field in WebSocketInputOperator should be volatile
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1894'>APEXMALHAR-1894</a>] -         Add an Input Port With An isConnected Method
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1922'>APEXMALHAR-1922</a>] -         [FileStreamContext] Set filterStream variable to transient
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1925'>APEXMALHAR-1925</a>] -         The kafka offset manager may not store the offset of processed data in all scenarios
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1928'>APEXMALHAR-1928</a>] -         Update checkpointed offsettrack in operator thread instead of consumer thread
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1929'>APEXMALHAR-1929</a>] -         japicmp plugin fails for malhar samples
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1934'>APEXMALHAR-1934</a>] -         When offset is unavailable kafka operator stops reading data
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1947'>APEXMALHAR-1947</a>] -         Ability to restrict number of messages per window by size for kafka input operator
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1949'>APEXMALHAR-1949</a>] -         JDBC Input Operator unnecessarily waits two times when the result is empty
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1960'>APEXMALHAR-1960</a>] -         Test failure KafkaInputOperatorTest.testRecoveryAndIdempotency
</li>
</ul>
            
<h2>        New Feature
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1812'>APEXMALHAR-1812</a>] -         Support Anti Join
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1813'>APEXMALHAR-1813</a>] -         Support Semi Join
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1904'>APEXMALHAR-1904</a>] -         New Kafka input operator using 0.9.0 consumer APIs
</li>
</ul>
    
<h2>        Improvement
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1895'>APEXMALHAR-1895</a>] -         Refactor Snapshot Server
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1896'>APEXMALHAR-1896</a>] -         Add Utility Functions For Working With Schema Tags
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1906'>APEXMALHAR-1906</a>] -         Snapshot Server support tags
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1908'>APEXMALHAR-1908</a>] -         Add Deserialization Function That Deserializes keys with multiple values
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1913'>APEXMALHAR-1913</a>] -         [FileSplitter] Need access to modifiedTime of ScannedFileInfo class
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1918'>APEXMALHAR-1918</a>] -         [FileSplitter] Need stopScanning method in Scanner
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1940'>APEXMALHAR-1940</a>] -         Create Operator Utility Class Which Converts Time To Windows
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1958'>APEXMALHAR-1958</a>] -         Provide access to doneTuple field in AbstractReconciler for derived classes
</li>
</ul>
            
<h2>        Task
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1892'>APEXMALHAR-1892</a>] -         Fix missing javadoc
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1905'>APEXMALHAR-1905</a>] -         Test the old kafka input operator is compatible with 0.9.0 broker
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1950'>APEXMALHAR-1950</a>] -         Identify and mark Operators and Components as @Evolving
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1956'>APEXMALHAR-1956</a>] -         Concrete generic Implementation of Kafka Output Operator with auto metrics and batch processing
</li>
<li>[<a href='https://issues.apache.org/jira/browse/APEXMALHAR-1964'>APEXMALHAR-1964</a>] -         [Checkstyle] Reduce the severity of line length check
</li>
</ul>