The following are changes currently implemented in development builds of Cyotek WebCopy. Please note that these changes are not final and may be removed prior to the next stable release. If you have any comments about these forthcoming changes, please contact us.

Nightly builds featuring these changes are available from the Downloads page.

Added

  • It is now possible to read additional URLs to scan from a text file [#281]
  • Added no-directories, max-redirect and header arguments to CLI
  • Added proxy, proxy-user and proxy-password arguments to CLI [#337]

Changed

  • Adding multiple URLs to scan is now easier using a free text field [#282]
  • Command line tools now report unknown parameters
  • Major reworking of internal decision making logic [#242]

Fixed

  • Only the last argument error was displayed when running command line tools
  • WebCopy will now retry URLs that fail with "The server committed a protocol violation" exceptions

The following changes are a sneak-peak of features that are currently being experimented with in separate branches from the core product. Due to this they are not available in nightly builds (and there is no guarantee they will be merged into the core branch).

Added

  • Crawl state persistence (pause, resume) [#165]

Changed

  • Crawling now uses breadth first when prioritising enqueued items
  • New WebCopy projects will default to enabling Brotli compression support
  • New WebCopy projects will default to saving headers
  • Major refactor of internal crawl logic, making it easier to test and to add new functionality

Download

Download current and archived versions of Cyotek WebCopy

Download

Minimum Requirements

Donate

Donate