Retrospective on Backup

Under the hood in Retrospect 7.7

Hi folk,

Sorry for the delays in responding to some of the great commentary on the last post. As you have seen, we've finally released Retrospect 7.7 (See our main site at http://www.retrospect.com/ for details). I'd like to spend some time today going over the changes under the hood and how that affected some of our decisions and why you see what you see in 7.7.

Retrospect 7.7 is a transition step between 7.6 and 8.0 in that it has the same cross-platform engine that ships with Retrospect 8.0 on the Mac, but with the older 7.6 UI. There were a lot of reasons for going with this approach. In the comments to the last post, there were folk urging us to get 7.7 out the door as soon as possible and others unhappy it wasn't going to be the full-blown new User Interface as the Mac product got. Given the resources we have today, not putting a brand-new UI saved us a lot of time, both in development and testing. At the same time, the new engine offers a lot of under-the-hood improvements, not least is the ability to run native on 64-bit machines.

There is definitely some pain involved with going with the new world order. As an example, the new engine uses the current standard of file paths on the Mac, so upgrading from 7.6 to 7.7 will cause us to want to back up Mac OS X clients from scratch. This won't affect future backups and the file path system will allow us to do hopefully cooler things in the future.

For more specific information, check out our forums at: http://forums.dantz.com/

JG

Sneak preview: Operations Log

Retrospect 8.1 is coming soon and I wanted to show you folk a sneak preview of one of the areas we've enhanced: the operations log.

The log in 8.0 was...functional, but there is not much to say after that. The new 8.1 log brings back the look and feel of the existing logs with icons and formatting:

And yes, the famous > for errors is also back.

We've also zipped up the find mechanism, Safari style:

Note that multiple matches all highlight - the current match is in gold, the others are white. Aside from the standard shortcuts (Command+G or Enter for next and Shift+Command+G or Shift+Enter for previous), we have also added wildcard and regular expression searching (in a single line of text).

Hope you are liking it!
JG

Search for Files Restore back

We haven't forgotten we owe you the second part of the scheduling UI walkthrough, but we are taking a brief break to cover a missing feature that has been added recently. Many of you have asked "What happened to Find Files restore?" This feature allows you to search across multiple media sets without the need to know the location of the file. Beta 5, now available at http://www.retrospect.com/products/software/retrospect8/publicbeta, adds this feature back into the product.

To restore using this feature, first launch the restore assistant:

The assistant's default is to restore files selected from a single point in time, but the third option allows you to search across all backups, even in multiple media sets.

Selecting "Continue" allows you to choose what media sets to look in and also what to search for. You can build complicated searches or, as in the example below, just look for part of a file's name.

Selecting "Continue" again prompts you for where you want to restore the files. The default is to create a new folder in the selected volume.

Selecting "Continue" causes Retrospect to start searching through all the media sets you selected, across every backup it knows about, and will display each media set that found a match. As you can imagine, this can take a while! Click "Preview" to see the files Retrospect has found and to choose the files you want to restore.

Here, we are only seeing one drive, "4G", that had any matching files. If multiple drives backed up to "Media Set A" had those files, they would show up in this tree as well. If the file has been backed up multiple times, you would see a different entry for each time it changed, with the modified date reflecting the date.

The final pane is the standard summary you see in the other assistant work flows. Selecting "Start Now" will begin the restore.

Hope you folk enjoyed the walk through and are also checking out the beta. Keep those comments coming!

Walkthrough of the new Scheduling UI

One of the areas we are looking to improve with the new UI is scheduling scripts. First time users are baffled and even long-time users are frequently amazed at finding some existing feature. Below, we will go through some screenshots to show off the new design. If you've downloaded the beta at http://www.retrospect.com/products/software/retrospect8/publicbeta, you can play along at home.

The following is the schedule tab for a newly created script:

 

Not much to do or see here except click the "+" icon. Doing that gives you the default schedule below:

The default schedule is to back up every week, Monday through Friday, starting at 10:00 PM. For backup scripts, you can choose the Media Set this schedule will go to via the Destination menu. Hopefully, it is pretty obvious that you can set the first start date via the calendar and the time each backup will occur by typing in the start field.

Clicking the menu next to the repeat label gives you the following options:

Never is useful for scheduling one-off actions, like doing a catch-up backup at the end of day. Hourly and daily allow you to schedule repeating backups, to occur every six hours for example, or every two days. This last option allows you to set up a rotating backup to back up to alternating media every day, as seen in the next screenshot.

The last repeat type, monthly, comes in handy where you may want to back up to an off-site media or take advantage of Retrospect's media actions to create a new Media Set before backing up or to recycle (overwrite) an existing Media Set.

The above screenshot shows the other media actions. "No media action" was formerly called "Normal" but that term confused too many novice users. This tells Retrospect to follow its default behavior: Start fresh if the Media Set is empty, otherwise append data to the media. Let us know if you like the new terminology. The menu next to "month(s)" allows you to pick other options like "on the second Saturday" or "on the last Saturday."

Finally, let's go back to the "weekly" repeat type and look at the last option:

 

For schedules that repeat on fixed days of the week, while we default to the normal approach of running the backup until it is complete, we also expose a feature that was formerly hidden in Script Options, the ability to stop a script at a fixed time. This is useful for making sure backups stop before the normal work-day begins. This ability to set backup windows is at the heart of Proactive Backup (Backup Server scripts in 6.1) and we will talk more about the scheduling interface for those scripts in our next post.

-JG

New Retrospect X screenshots!

After being out of the office at Apple's Sales Engineer Fall Camp up in Tahoe City on Wednesday and Thursday, I didn't have time to get the new screenshots up yesterday, so here they are (click any screenshot to see a full-size version)...

Dashboard/Reports
This is the main dashboard view showing a single Retrospect server in the left-nav. Retrospect is showing the four reports that I saved from various other views. Reports are like smart folders; you can save a set of rules for each one. Clicking the Edit button on the right side of each report's title bar takes you to the corresponding view where you can make changes to the criteria you've set.

You will notice that several of the reports have green check marks in their leftmost column. These are placeholders for a variety of status icons that will be added in a future build. Status icons will allow a user to tell at-a-glance if an activity is running, has generated an error or warning, etc.

Adding Clients
Neither Retrospect 6.1 for Mac nor 7.6 for Windows do a very good job today of dealing with long lists of network clients. They only provide alphabetical sorting, and they don't really provide any information other than client name and computing platform. As you can see in this screenshot, Retrospect X will not only show a variety of information for each client computer, but the view can be sorted by any column. Here, we're sorting by computing platform as an example.

New to the Mac with Retrospect X is the ability to define multiple network interfaces, so you could have, say, one set up for either side of a firewall, or you could keep backup traffic from crossing subnets.

You'll also likely notice that apostrophes aren't yet displaying properly. That's because we're still making significant changes to the UI and engine, but we'll soon begin the work of fine-tuning and refining.

Tagging Sources
The Sources view shows all local volumes, client computers and their volumes, as well as network-attached storage devices or volumes accessed through some file sharing protocol (instead of the Retrospect Client software). Also visible in this view are favorite folders defined by the administrator, which replace Retrospect's "subvolume" moniker. Again, this is a fairly rough view, but there are several key things to point out...

At the very top of the Sources list are several scope buttons: Servers, Desktops & Laptops, Local, Client, and Shares. Clicking these buttons will cause the Sources list to be filtered according to the type of operating system installed on a source's host computer, or according to how the Retrospect server is connecting to that source.

Below the scope buttons are sortable columns, with only a portion of the available columns shown. By control/right-clicking on any column heading, users can turn the available headings on or off to customize the view as needed.

Below the Sources list, in the Options pane, you can see that we've assigned two tags to Robin's MacBook: Building2 and Critical Systems. Tags replace the Source Groups container in previous editions of Retrospect. Source Groups were essentially folders containing aliases/shortcuts to whichever sources/volumes the user assigned. They provided a level of abstraction and an ability to logically group sources for backups. For example, one might have a Notebooks Source Group that contained aliases to all the laptop computers they were backing up with Retrospect, and then they would just point their backup script at the Notebooks Source Group. With Retrospect X, tags provide a more flexible and easily recognizable method for doing the same thing.

Scheduling a Script
This screenshot shows the interface for scheduling a script, the Daily Backup script in this case. Just as with previous versions of Retrospect, scripts can contain multiple schedules and target multiple destination media sets, so you only need a single script for each group of sources to which you want the same schedule to apply

To add a schedule, one simply clicks the + button in the Schedule pane and selects the various options in the Details pane below. Here, our backup will be targeting Test Set A as its destination media set, and no special media action will be performed. Media actions tell Retrospect what to prior to starting the actual backup. "No media action" (like "normal" today) means that Retrospect will start writing to empty media if no backup has yet been performed to the chosen media set, or, if data is already contained in the destination media set, then Retrospect will append to the existing media. Retrospect can also be told to skip to new media, recycle/erase existing media, or to archive the existing media set and start a new one.

Retrospect X gets a calendar that makes it easier to select a script's start date, and it also gains a stop time option, where the administrator can force Retrospect X to stop backing up at a certain time.

I hope that everyone finds these new screenshots and accompanying text helpful as far as seeing where we are with Retrospect X. We certainly appreciate your feedback here on the blog, and we invite you to share your thoughts in the comments section.

The real test will be getting a beta in your hands, which we're working towards as hard as we can. Thanks for reading—I'll post another update soon.

-Eric

RetroSpeak (or, "What the heck is a Subvolume?")

I've been up to my eyeballs in Retrospect terminology this week, as we're making decisions on what specific terms will be used in the UI and user documentation. Retrospect uses...uncommon terminology for some things, in both the Mac and Windows versions. In fact, Retrospect uses so many unique terms, or different meanings for common terms, we jokingly refer to the use of such as "RetroSpeak".

We keep several goals in mind as we go through the terminology process:

1. Choose explanatory/descriptive terminology whenever possible.
2. If a new term is not significantly better than the current term, we probably shouldn't make the change.
3. Avoid changing terms that will confuse/hinder existing Retrospect customers.
4. Use generic terms that will work across Windows, Mac, etc.
5. Call out Retrospect's technology, but don't overdo it.

These are good guidelines, and they definitely help us with most decisions, but some aren't as clear as others, and external input is just the thing we need right now.

Script/Job/Task
Today, in both the Mac and Windows versions of Retrospect, a saved backup/duplicate/restore/whatever operation is called a "script". A script includes one or more sources, destinations, options, and usually at least one schedule. While Retrospect is designed for use primarily at small and medium businesses, it is often used by individuals who aren't IT pros, and the term script can be intimidating (i.e., some people relate that to a scripting language like AppleScript or VB). Most other backup software calls these saved operations "jobs", but I personally have some sort of allergy to that term, and just try to come up with an icon for it.

The term that seems to make the most sense is "task", but I don't think it passes Rule #2 above. Your thoughts?

Proactive Backup/Backup Server/Policy Backup
One of Retrospect's cool technologies that has been around for a long time is its ability to go down a list of sources and back them up to available media based on priority, with the least-recently backed up sources having the highest priority. Version 6.1 on the Mac calls these "Backup Server" scripts, while they're called "Proactive Backup" scripts in version 7.6 on Windows.

We've put some time and effort into branding/marketing Proactive Backup, and it's certainly more descriptive than Backup Server, but "Policy Backup" or "Policy-based Backup" might be even more descriptive. Still, we're leaning towards using Proactive Backup. Your opinion?

Thanks for the feedback, and please keep it coming!

Happy Halloween, all.

-Eric

We're working with Iconfactory!

If you saw the early screenshots of Retrospect X that I posted a couple of months ago, you likely noticed our placeholder icons. As a UI element, icons play a huge role in Mac apps. Finder icons in Leopard scale up to 512x512 pixels, and even toolbar icons can be gorgeous.

When we had our first UI feedback meeting at Apple, we showed a Retrospect X interface where all the icon placeholders were Indiana Jones icons from Anthony Piraino and the Iconfactory. It was pretty funny, actually, even though it made no sense whatsoever.

We definitely know how important it is to have good-looking icons in Retrospect X that best represent the concepts and information they need to convey to the user, so I am pleased to say that we are going to be working with the Iconfactory's design team to come up with our final artwork. They do great work, and we're thrilled to have their help.

Expect to see some of their work here in the coming weeks.

-Eric

A great day at Apple

Yesterday, JG Heithcock (director of engineering and primary software architect for Retrospect), Brian Dunagan (lead Retrospect UI engineer), and I (lead Retrospect windbag...er...product manager) had our third UI review meeting at Apple. While the exact details of the meeting are confidential, I can say that Apple's response to the latest iteration of Retrospect X's UI was highly favorable. As always, we walked away with some gems of advice, but the tweaks were only minor this time.

I think you're going to like what we've done, and I can hardly wait to show you. We'll get some new screenshots posted soon.

-Eric

Requests for features that are already in Retrospect

One of the comments on a previous post reminded me of one of the best examples that a product's UI just ain't cutting it: when folks consistently request a feature that's already in the product.

When it happens only once or twice, you figure the problem is somewhere between the user's chair and their computer. But when you hear such requests with a fair level of regularity, well, it's time to suck it up and admit the problem is in the UI, at least for a significant enough percentage of users that something should be done.

What's new in Retrospect: Reporting

Eric's last blog entry gave a glimpse of some of the new reporting features. I'm going to go through that in more detail in this entry. As Eric warned earlier, the user interface is still in progress and subject to change. For example, the icons are currently just conceptual placeholders.

Reporting in current versions of Retrospect, while useful, has a number of limitations:

• Finding things in a report was cumbersome.
• They either didn't give enough information, or the information was hidden in another area.
• There wasn't much to do with the information once you had found it.
• You could only change the report in specific, pre-defined ways.

Finding things
You could always do a text search in all the lists and reports in Retrospect, and that is not going away as evidenced by the now prevalent Search Bar visible in the global toolbar. To further narrow a search, we are adding Scope Bars which allow you to quickly focus in.

Sometimes that isn't enough, however. So in the new Retrospect, we are introducing advanced find features. These will work very much the way finding in a Finder window works: You can keep adding layers by clicking the plus icon after doing a search, or go there directly by choosing Find from the Edit menu.

More information
Today, most lists in Retrospect only have one or two columns. In the new version, we are exposing all object properties as possible columns. Many of these are hidden by default but can be shown by right-clicking the column header (as in iTunes) or by choosing Options from the View menu (as in the Finder). You can also use these in the advanced find mode above.

In the limelight: Backups and Activities
Currently, the Backup Report lists backups of sources in a rather rigid way. It only allows two different sets of columns and interaction with the report is limited. You can't browse the backup, nor see anything other than the most recent backup.

In the Windows version of Retrospect, if you wish to see backups previous to the ones shown in the Backup Report, or you wish to know the number of files backed up or other data than what is shown, you can go to the History tab in the Activity Monitor. But the two are not integrated so double-clicking an item in the Backup Report only shows you the script that initiated the backup. Moreover, the History tab can’t be customized in any way, so you cannot choose to only view unsuccessful backups or any of the other reports that are readily available in the Backup Report.

In Retrospect X, we will be bringing over the contents of the Activity Monitor from the Windows version and promoting it to the left sidebar as a single global list. All properties in the exported backup report—as well as in the history report—will be available columns that you can show and sort as needed. The existing Backup Reports will be converted to use this global list and we will be adding others. If we leave anything out, you will be able to create your own reports to see only failed backups, backups with poor performance, backups of critical machines or perhaps backups to specific Backup Sets.

Status Icons
One of the comments I hear from many people is how verbose Retrospect is. We really go out of our way to describe everything. Sometimes less is more, so most views will now have status objects. This is really exposing information we already know and report on, but hopefully will do a better job calling out what is going on.

Reports as a launching pad
Another comment we hear a lot is that after you finally do find that user that hasn't been backed up or that a backup set that is low on free space, you now have no clue where you need to go to fix the problem. Enter launch buttons, inspired by the icons in iTunes that take you to the iTunes music store or publish a playlist. In our case, clicking the arrow next to "Steve's Air" will take you to the Sources list and select "Steve's Air" from the list. At that point, all the actions you can take to find out why Steve isn't showing up on the network are in a toolbar at the bottom of the window. Alternately, you can just click on the name of the object and it will take you there as well, à la clicking the labels in Apple's Address book.

Creating a new report
So far, we haven't made or looked at any actual reports. This can be done by clicking the Save button visible once you have done a search.

Selecting the Save button brings down a sheet that allows you to give the report a name and optionally add it to the Retrospect dashboard. Saved reports show up under the Reports icon in the sidebar on the left. The report will keep the currently shown columns and sorting order.

Viewing and changing a report
Selecting a report under the Reports icon or double-clicking its title in the Retrospect dashboard will open the report for viewing. You can search in the report using the same search bar as the original view. Reports are dynamic and change as Retrospect continues to back up and run. If you make any changes, you can select "Save" to save your changes or choose "Save as..." to create a new report.

Reports out of the box
In addition to the reports Retrospect offers today, we will be adding the following new ones:

• Not backed up
• Not backed up recently (in 3 days)
• Need new media
• Active Media (Media that had been backed up to within the month)
• Inactive Media (Media that has not been backed up to within a year)
• Media free space (only disk, columns of free space)

What do you all think of these? Any others that you would like to see? Let us know!

JG