Cyotek WebCopy Upcoming Changes Copy websites locally for offline browsing
The following are changes currently implemented in development builds of Cyotek WebCopy. Please note that these changes are not final and may be removed prior to the next stable release. If you have any comments about these forthcoming changes, please contact us.
Nightly builds featuring these changes are available from the Downloads page.
Due to changes in how WebCopy determines whether or not to process a given URL there could be differences with how WebCopy 1.9 works against previous versions. Please report any inconsistencies to us!
- It is now possible to read additional URLs to scan from a text file [#281]
headerarguments to CLI
proxy-passwordarguments to CLI [#337]
input-fileargument to CLI [#282]
- Adding multiple URLs to scan is now easier using a free text field [#282]
- Command line tools now report unknown parameters
- Major reworking of internal decision making logic [#242]
- New WebCopy projects will default to enabling Brotli compression support
- New WebCopy projects will default to saving headers
- Only the last argument error was displayed when running command line tools
- WebCopy will now retry URLs that fail with "The server committed a protocol violation" exceptions
- If using the default user agent, WebCopy will now try a default browser agent if a 401 response is returned when validating the URL [#382]
- When issuing a 401 challenge, WebCopy could include additional header information in the realm field
The following changes are a sneak-peak of features that are currently being experimented with in separate branches from the core product. Due to this they are not available in nightly builds (and there is no guarantee they will be merged into the core branch).
- Crawl state persistence (pause, resume) [#165]
- Crawling now uses breadth first when prioritising enqueued items
Download current and archived versions of Cyotek WebCopyDownload
- Windows 10, 8.1, 8, 7, Vista SP2
- Microsoft .NET Framework 4.6
- 20MB of available hard disk space
This software may be used free of charge, but as with all free software there are costs involved to develop and maintain.
If this site or its services have saved you time, please consider a donation to help with running costs and timely updates.Donate