This content has moved - please find it at https://devblog.cyotek.com.

Although these pages remain accessible, some content may not display correctly in future as the new blog evolves.

Visit https://devblog.cyotek.com.

Introducing CopyTools 2.0

Out of our product range, my own personal favourite is CopyTools. This program has been working in the background for years quietly ensuring every digital file of import is backed up, and even performing some maintenance tasks such as downloading Azure SQL databases.

As with most software it is by no means flawless, and recent (internal) builds compounded the issue rather than improving it. Although it was originally supposed to support remote sources such as FTP servers and Azure blob storage, the model was too rigid and in order to add this support would require massive changes to program's core, instead of supporting the extensibility model that most Cyotek products offer.

As a result of this, for the last few months we've essentially ignored all other products while we worked on what has now become CopyTools version 2 and we can now offer the first alpha build of version 2.0.

While it's flagged as alpha, I can state that it has been running in production for a month now without major incident. But still, given the purpose of CopyTools, I feel an "alpha" status is appropriate at this stage, especially as I haven't really retested all the UI options.

Again, just to be clear - this is an alpha build. Just because I'm running it in production doesn't mean you should. Here be dragons!

What's new

The user interface, by and large, remains the same - there small changes here and there, but it's pretty much the same as before. A lot of the work involved rewriting the core engine along with a huge effort in expanding the test suite.

The main new feature of this build is end point support. As I mentioned in my intro, originally CopyTools was supposed to support local file copies and FTP. Instead it ended up being local file support and compressed archive support bolted on top of that. Now however, end points are properly defined, and are not tied to the core engine, meaning they can be replaced at will.

New options for retrying files that are in use were also added, but there's no UI exposure yet so these options are currently unavailable.

FTP

CopyTools now includes basic FTP support. This new end point is bi-directional, which means means you can easily copy from an FTP server to a local system, local files to an FTP server, or even FTP server to FTP server.

Command line execution

We've also included a command line client for previewing or executing profiles. This was initially written solely to provide a quick means of running the engine for performance/memory profiling scenarios, but is now part of the core product and can be used in scripting or task scenarios.

Getting the alpha

You should uninstall version 1 prior to installing version 2 otherwise obsolete program files are left behind

You can download the alpha from the Nightly Builds page.

CopyTools 2.0 stores profiles in a different file (profiles.xml) from version 1 (profiles.ctp). When installing the alpha, your profiles will be automatically migrated, leaving the original profiles intact - allowing you to easily revert back to version 1 if required

After installing CopyTools, the setup program will launch the program with the poller suspended. Before either resuming the poller, or restarting CopyTools you should check that all profiles were successfully migrated. For profiles using inclusion/exclusion masks, it is also strongly recommended you preview the profiles first to ensure the masks operate as expected (see Known Issues for more details).

Known issues

Help files

Help files haven't been updated for this build.

Profiles using inclusion/exclusion masks

The method by which inclusion/exclusion masks are processed has been changed. As a consequence, it is possible that there could be subtle differences and files that were previously excluded may not now be and vice versa. We strongly recommend that you preview any profiles using inclusion/exclusion masks and check the preview looks correct.

If the preview does not look correct and is including (or excluding) files that it shouldn't be, please contact us, including the full path and name of the file, and the relevant mask.

Profile flags

The use multi thread directory scanning flag is currently ignored.

Command line client

If the command line client is used to run a profile while the GUI client is running, the GUI client will not update to reflect the last run and outcome. It will also overwrite that information when it next runs any other profile.

In addition, if both the CLI and the GUI try to run a profile that is using the same log file, one of the processes will fail.

"Access Denied" using Windows Vista

Now that we no longer support Windows XP, we have Windows Vista VM's as a "lowest common denominator" for running smoke tests. When trying to preview/copy profiles on Vista where reparse points are involved, an "Access Denied" error message will appear, even if the option to skip reparse points is enabled. And of course the Visual Studio 2015 remote debugger doesn't support Vista, so I'll have to fix this one a little later.

What's next

Future builds should address some other shortcomings, such as adding new copy modes and finally sorting out the scheduling - keep an eye on the upcoming changes page for details as and when they occur.

Update History

  • 2016-11-29 - First published
  • 2020-11-23 - Updated formatting

About The Author

Gravatar

The founder of Cyotek, Richard enjoys creating new blog content for the site. Much more though, he likes to develop programs, and can often found writing reams of code. A long term gamer, he has aspirations in one day creating an epic video game. Until that time, he is mostly content with adding new bugs to WebCopy and the other Cyotek products.

Leave a Comment

While we appreciate comments from our users, please follow our posting guidelines. Have you tried the Cyotek Forums for support from Cyotek and the community?

Styling with Markdown is supported