Release Notes - ASF JIRA

Release Notes - Nutch - Version 2.3.1 - HTML format

Configure Release Notes

Sub-task

[NUTCH-1169] - Write JUnit tests for urlfilter-prefix
[NUTCH-1923] - Nutch + Cassandra Docker
[NUTCH-1924] - Nutch + HBase Docker

Bug

[NUTCH-1572] - Nutch 2.x should use o.a.g.mem.store.MemStore for testing
[NUTCH-1679] - UpdateDb using batchId, link may override crawled page.
[NUTCH-1893] - Parse-tika fails to parse feed files
[NUTCH-1922] - DbUpdater overwrites fetch status for URLs from previous batches, causes repeated re-fetches
[NUTCH-2009] - Fetcher does not work with batchID
[NUTCH-2019] - ClassPathException sending topN argument for /job/create using Nutch 2.x RESTApi
[NUTCH-2028] - java.lang.IllegalArgumentException: can't serialize class org.apache.avro.util.Utf8
[NUTCH-2029] - Mark.checkMark returns empty string when null is expected with mongodb storage
[NUTCH-2042] - parse-html increase chunk size used to detect charset
[NUTCH-2045] - index-basic incorrect assignment of next fetch time (page.getFetchTime()) as page fetch time
[NUTCH-2080] - Eclipse compilation issue
[NUTCH-2094] - Stopping and Restarting a crawl has issues in the Web UI
[NUTCH-2101] - Upgrade Nutch 2.X to Hadoop 2.5.1
[NUTCH-2130] - copyField rawcontent creates error within schema.xml
[NUTCH-2143] - GeneratorJob ignores batch id passed as argument
[NUTCH-2168] - Parse-tika fails to retrieve parser
[NUTCH-2377] - Nutch can't parse relative links

New Feature

[NUTCH-1900] - DockerFile for Nutch 2.x
[NUTCH-1941] - Optional rolling http.agent.name's
[NUTCH-1944] - Add raw content to indexes
[NUTCH-2105] - Update Nutch Cassandra Dockerfile to work with Gora Nutch 2.3.1

Improvement

[NUTCH-1062] - Migrate BasicURLNormalizer from Apache ORO to java.util.regex
[NUTCH-1286] - Refactoring/reimplementing crawling API (NutchApp)
[NUTCH-1920] - Upgrade Nutch to use Java 1.7
[NUTCH-1925] - Upgrade Tika to version 1.7
[NUTCH-1946] - Upgrade to Gora 0.6.1
[NUTCH-1981] - Upgrade icu4j
[NUTCH-1990] - Use URI.normalise() in BasicURLNormalizer
[NUTCH-1994] - Upgrade to Apache Tika 1.8
[NUTCH-2018] - Ensure that the Docker containers for Nutch 2.X are part of the Release Management Documentation
[NUTCH-2050] - Upgrade HBase and Hadoop versioning on 2.X HBase Docker
[NUTCH-2077] - Upgrade to Tika 1.10
[NUTCH-2082] - Upgrade to Apache Tika 1.10
[NUTCH-2107] - plugin.xml to validate against plugin.dtd
[NUTCH-2169] - Integrate index-html into Nutch build

Task

[NUTCH-1886] - Review and update default.properties
[NUTCH-1936] - GSoC 2015 - Move Nutch to Hadoop 2.X

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.

Release Notes - Nutch - Version 2.3.1
    
<h2>        Sub-task
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1169'>NUTCH-1169</a>] -         Write JUnit tests for urlfilter-prefix
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1923'>NUTCH-1923</a>] -         Nutch + Cassandra Docker
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1924'>NUTCH-1924</a>] -         Nutch + HBase Docker
</li>
</ul>
            
<h2>        Bug
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1572'>NUTCH-1572</a>] -         Nutch 2.x should use o.a.g.mem.store.MemStore for testing
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1679'>NUTCH-1679</a>] -         UpdateDb using batchId, link may override crawled page.
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1893'>NUTCH-1893</a>] -         Parse-tika fails to parse feed files
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1922'>NUTCH-1922</a>] -         DbUpdater overwrites fetch status for URLs from previous batches, causes repeated re-fetches
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2009'>NUTCH-2009</a>] -         Fetcher does not work with batchID
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2019'>NUTCH-2019</a>] -         ClassPathException sending topN argument for /job/create using Nutch 2.x RESTApi
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2028'>NUTCH-2028</a>] -         java.lang.IllegalArgumentException: can&#39;t serialize class org.apache.avro.util.Utf8
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2029'>NUTCH-2029</a>] -         Mark.checkMark returns empty string when null is expected with mongodb storage
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2042'>NUTCH-2042</a>] -         parse-html increase chunk size used to detect charset
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2045'>NUTCH-2045</a>] -         index-basic incorrect assignment of next fetch time (page.getFetchTime()) as page fetch time
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2080'>NUTCH-2080</a>] -         Eclipse compilation issue
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2094'>NUTCH-2094</a>] -         Stopping and Restarting a crawl has issues in the Web UI
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2101'>NUTCH-2101</a>] -         Upgrade Nutch 2.X to Hadoop 2.5.1
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2130'>NUTCH-2130</a>] -         copyField rawcontent creates error within schema.xml
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2143'>NUTCH-2143</a>] -         GeneratorJob ignores batch id passed as argument
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2168'>NUTCH-2168</a>] -         Parse-tika fails to retrieve parser
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2377'>NUTCH-2377</a>] -         Nutch can&#39;t parse relative links 
</li>
</ul>
            
<h2>        New Feature
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1900'>NUTCH-1900</a>] -         DockerFile for Nutch 2.x
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1941'>NUTCH-1941</a>] -         Optional rolling http.agent.name&#39;s
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1944'>NUTCH-1944</a>] -         Add raw content to indexes
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2105'>NUTCH-2105</a>] -         Update Nutch Cassandra Dockerfile to work with Gora Nutch 2.3.1
</li>
</ul>
    
<h2>        Improvement
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1062'>NUTCH-1062</a>] -         Migrate BasicURLNormalizer from Apache ORO to java.util.regex
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1286'>NUTCH-1286</a>] -         Refactoring/reimplementing crawling API (NutchApp)
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1920'>NUTCH-1920</a>] -         Upgrade Nutch to use Java 1.7
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1925'>NUTCH-1925</a>] -         Upgrade Tika to version 1.7
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1946'>NUTCH-1946</a>] -         Upgrade to Gora 0.6.1
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1981'>NUTCH-1981</a>] -         Upgrade icu4j
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1990'>NUTCH-1990</a>] -         Use URI.normalise() in BasicURLNormalizer
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1994'>NUTCH-1994</a>] -         Upgrade to Apache Tika 1.8
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2018'>NUTCH-2018</a>] -         Ensure that the Docker containers for Nutch 2.X are part of the Release Management Documentation
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2050'>NUTCH-2050</a>] -         Upgrade HBase and Hadoop versioning on 2.X HBase Docker 
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2077'>NUTCH-2077</a>] -         Upgrade to Tika 1.10
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2082'>NUTCH-2082</a>] -         Upgrade to Apache Tika 1.10
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2107'>NUTCH-2107</a>] -         plugin.xml to validate against plugin.dtd
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-2169'>NUTCH-2169</a>] -         Integrate index-html into Nutch build
</li>
</ul>
            
<h2>        Task
</h2>
<ul>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1886'>NUTCH-1886</a>] -         Review and update default.properties
</li>
<li>[<a href='https://issues.apache.org/jira/browse/NUTCH-1936'>NUTCH-1936</a>] -         GSoC 2015 - Move Nutch to Hadoop 2.X
</li>
</ul>