Recently, the 1TB LaCie Network Space drive, as ridiculed previously, vanished off the network.  The router wasn`t showing the port connected, and nmap-ping the LAN shows the device had vanished.  I did the usual routine of swapping Ethernet cables, trying another port in the router, etc. but no go.  Even after connecting the drive to the laptop over Ethernet, the beast had vanished.

I ripped the pathetic piece of pain out of its NAS enclosure and chucked it in a USB to see what Linux made of the situation.  The drive, and every one of the ext3/xfs partitions is healthy, yet the device is plopped.  Looks like the Ethernet port just gave up.  No offence, but that realllly shouldn`t happen with a silly little drive that just sits beside the router all day long.  Turns out the drive itself was a little Samsung 4200rpm model.  Which might explain why it was always shit slow.  I`d always assumed it was 7200rpm.  Hey ho.

LaCie – that`s the first and last time I`ll buy any of your products.  Awful.  2/10.  You lost another point thanks to Neil Poulton, whoever he is and your philosophy of design over reliability.  Perhaps his name is an anagram of “overpriced, underperforming, zero reliability”.

Back to the rsync scripts again – I`ve nuked the drive as NTFS and mounted it as the same drive letter on Windows, so the rsync scripts just work.  Probably the first time ever on Windows anything has ever “just worked”.

Tonight, I decided to restore my (jailbroken) iPhone.  A few of the bits and bobs I`d installed were slowing the phone down to the point of it being annoying.  I`d fairly recently encrypted the backups iTunes insists on taking everytime you plug the phone in, and … well, I forgot the sodding password.  Or so I thought.

So for the last 3 hours, I sat and worked with my “password generator” (i.e. brain) thinking through the various combinations of stuff I use to come up with passwords, and tried every combination possible.  Each time, I was told that the password was wrong.  I must have gone through variations of 40-50 passwords and nothing worked.  Incorrect password.  Does not compute.  Negatory.  Zero zero zero one one one zero.

I shut down iTunes after a while.. plonked about on other stuff, then thought I`d have another bash.  And suddenly the phone`s restoring itself with no input.

Yes folks.. there is a really nasty bug in there somewhere that means even valid passwords aren`t making it through.  If I haven`t made clear in previous posts, I think iTunes is a bloody awful piece of software.
Christ.

As for this new iPad Apple have released… I can`t help think they`ve missed a trick there by sticking the iPhone / iPod Touch OS on there.  Roll on the new devices from Acer and so on – I won`t mention Asus as my personal experience of their hardware has been absolutely shocking.  A bit of competition and maybe Apple will stop locking the hell out of their devices.  Even Microsoft dropped the idea of limiting the number of simultaneous applications running from their cutdown versions of Windows.

Using the DeltaCopy Server to Keep The Drives In Sync – Using Deltas!

Once again, being not very happy with the previous not-quite-right way of doing things, I had a little poke, a nudge, and a further poke at DeltaCopy and its configuration to try to get the rsync working as rsync should work.  Ideally, you`ll want to use rsync through a server, and in this case the rsync server is DeltaCopy, or specifically, the DeltaCopy server installed with the application.  The previous post provides a method to implement a simple “if the file or directory exists, check it`s up-to-date, and if not, re-copy it; if it doesn`t exist, copy it”, but one of rsync`s biggest features is that of copying deltas, hence the name of the DeltaCopy tool.  The deltas are differences in files, and rsync will attempt to copy only changes in files to bring them up to date, thus saving in the amount of time required to copy.  Now, copying deltas can be oversold (depending on the type of file, it`s not always possible just to apply the changes, so a full re-copy is necessary) and it is more resource intensive, but for a large number of files, particularly text files (HTML, log files, code, etc.) can produce a massive speed up in the sync time.

rsync-ing the Photos Directory

Here`s the scenario, if you`ve already forgotten, or can`t be arsed reading the previous post:

USB drive mounted on U:
Lacie NetworkSpace network drive mounted on M:

U: is my primary work drive with all my gubbins on it, and M: is a network-attached drive for keeping a backup of everything.  I don`t always want to re-sync everything, as even with rsync, just finding the files and directories in order to test whether or not they`re up to date can take ages.  What we`ll do here is just rsync the photos directory, U:\photos to M:\photos.

The file we`re interested in here is deltacd.conf, located if you`ve installed using defaults, to

C:\Program Files\Synametrics Technologies\DeltaCopy\deltacd.conf

Open this file in Notepad and take a look.  What we see here are a number of “modules” which can be source or destination locations.  We can`t use a source module and a destination module if they`re both going through the server.  So we need to specify a module pointing to the U drive, and hack our little batch file to point to the M drive.

Here`s my module representing the photos directory on the U: (USB-attached) drive:

[usbphotos]

path = /cygdrive/u/photos/
comment = photos on usb drive
read only = false
auth users =
secrets file =

You can see this just points to U:\photos using the cygwin notation.  In order to make use of this, I then edit my little batch file, backup-photos-to-network.bat:

“C:\Program Files\Synametrics Technologies\DeltaCopy\rsync.exe” –log-file=”/cygdrive/c/rsync-photos-out.txt” -h –progress –delete-during -v -v -rlt localhost::usbphotos// “/cygdrive/m/photos”
pause

So, same parameters as before, but different source and destination.  The localhost::usbphotos// tells rsync to talk to a rsync server (“daemon”) on localhost and use the usbphotos module to grab its actual location, which of course is /cygdrive/u/photos AKA U:\photos  There`s also a pause in there to keep the DOS prompt open when the job completes – always nice to be able to see the status, else the DOS prompt will vanish after completion, which won`t be helpful if you`ve an error somewhere in the batch file.

One thing you`ll notice when using rsync deltas is that you`ll see a lot more CPU usage on your machine.  However, assuming you see uplift from the delta copying, this can make it all worthwhile.  Also, if you`ve a lot of files, it can be worth specifying a different log file for each job as the files can grow very large very quickly.  Finally, if you`re that way inclined, you can use the nasty Windows task scheduler to schedule regular runs of these batch files to keep everything up to date on the network drive.

That concludes my quick and nasty DeltaCopy rambling; basically I found the GUI too awkward for something I wanted to do quickly, and then ended up bypassing it completely to do what I needed to do.  I now however have a lovely mirror of my files.  Now to find somewhere to hide the drive!  I`ve been thinking about online file storage, but currently trying to keep 600GB and counting online would take too long to sync, and it`s just easier to have another drive, although that`s of little comfort if the house burns down.

Output From The Batch File

C:\>”C:\Program Files\Synametrics Technologies\DeltaCopy\rsync.exe” –log-file=”/cygdrive/c/rsync-work-out.txt” -h –progress –delete-during -v -v -rlt localhost::usbwork// “/cygdrive/m/work”
opening tcp connection to localhost port 873
sending daemon args: –server –sender -vvltre.iLs . usbwork//
receiving incremental file list
delta-transmission enabled
./
DSCF6251.JPG is uptodate
DSCF6261.JPG is uptodate
DSCF6277.JPG is uptodate
Smarty-2.6.22.zip is uptodate
Thumbs.db is uptodate
databases/headstaggers.mdb is uptodate
databases/headstaggers_gallery.txt is uptodate
databases/hscv.sql is uptodate
databases/hscv.sql_original_test is uptodate
databases/mydyson.txt is uptodate
databases/mydyson.txt.save is uptodate
databases/v.jsp is uptodate
[...]

I obviously need to do a bit of clean up!  Again, if you`re playing around and testing, avoid using –delete-during until you`re sure things are working!  “YMMV” as they seem to say.

Backing Up Local USB Drive to Network Drive Using Windows and Stinky Batch Files

Well, after much arsing around, I finally found a half-way useful method to backup my plethora of rubbish to/from the Lacie NetworkSpace.  This really bugged me, what with the none-too-nippy performance of the NetworkSpace, and its abysmal backup functions.  Also, using Windows around the house can be a real pain; Unices make this stuff so much easier but I really wanted a quick Windows method to perform the file-by-file backup.

So back to the setup: 1TB Lacie NetworkSpace drive, and 1TB Toshiba USB number, one drive (Toshiba USB) to be used as a primary lump of storage, and one (NetworkSpace) as a backup.  Initially I had the Lacie mounted as a Windows network drive, with the Toshiba plugged into the Lacie`s USB port, with the intention of kicking off a backup every week or so, depending on how much extra rubbish I plonked on the Lacie.  The Lacie`s backup utility is simply awful – read the other post if you`re interested, and with the extra drop in performance from running the drive over the network, I had a change of plan.

Now I run the Toshiba over the laptop`s USB connection, something I`m not over the moon about as it limits portability, but I get the performance and means iTunes doesn`t lock up everytime I try to access a file.  That just leaves syncing up the data so I still have a useful backup.  Having taken a look online for a useful tool and coming up blank, I came up with a half-sensible, half-hacky solution.

The setup consists of:
Drive U: usb connected Toshiba 1GB drive
Drive M: network mounted Lacie NetworkSpace drive

For trivia, I chose “U” for “USB drive”, and M for “media drive” as the Lacie was originally to be just for music and the like; it kinda stuck when I plonked all of my other bits and bobs on there.  The purpose of the mission was to keep M: (the backup) up to date with U: the 1TB goody bag of my photos, work, and other rubbish.

Had I been running Linux all the time, a simple rsync install on the Linux box would have done the job nicely.  For anyone who`s never used rsync, it`s a really nice way to synchronise files between a source and a target, whether they be drives, directories (“folders”), or just individual files.  However, over in Windows land, everything tends to be clicky-click button driven, so I was looking for something of that ilk, more out of laziness than lack of technical nouse.

I came across one tool that sort of did the job for me, DeltaCopy, available from http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp

Using the DeltaCopy GUI, which to be honest, was… a bit of a pain, I ran a full sync from the Lacie drive to the Toshiba drive to get the Toshiba drive (which had started out empty) up to date.  This took blummin` ages.  For anyone who`s never used rsync to sync a large amount of data (~600GB in this case), this can take blummin` ages.  When this succeeded (after a few days), I tried to set up a reverse sync from the Toshiba to the Lacie, i.e. from the fast, locally connected drive, to the network drive.  This is where problems crept in including lots of “code 5″ “chdir failed” errors.  So back to [insert search engine of choice here] I went to see if I could just sort out DeltaCopy.  I gave up after a little hacking – seems a lot of people encountered the same problem, “nads to that”, thought I.

DeltaCopy is an interesting tool, and is essentially a build of rsync for Windows, along with some necessary libraries from cygwin (a really nice UNIX-compatible environment for Windows), and a GUI on top.  It`s also usable from the command line – I wanted to avoid this just out of avoiding hassle, but eventually submitted.

Sample rsync Batch Files

So instead of making use of the GUI, I opened Notepad, that trusty Windows text editor (…) and hacked together a bunch of quick scripts to rsync specific directories over to the network drive.  I won`t always want to backup everything, and may just want to backup a bunch of photos I`ve recently copied onto the USB drive.

Backing up the whole drive, backupall.bat

“C:\Program Files\Synametrics Technologies\DeltaCopy\rsync.exe” –log-file=”/cygdrive/c/rsync-out.txt” -h –progress –delete-during -v -v -rlt “/cygdrive/u/” “/cygdrive/m/”

So, what`s going on here?
“C:\Program Files\Synametrics Technologies\DeltaCopy\rsync.exe” is the name of the program we`re running, including the full path on disk (i.e. where it`s been installed); it needs double quotes as there are some spaces.  That entire mess is just to run one command.  Everything that appears after it are options.

–log-file=”/cygdrive/c/rsync-out.txt”
I wanted a log file to check when the job`s been done, to check for errors, etc.  You`ll see the /cgydrive/c/ stuff, which is interesting.  Under the hood, cygwin can see your local and network drives if prefixed with /cygwin.  In this case, we are referencing the C: drive, which cygwin can access as /cygdrive/c -  so the log file will be plonked onto C:\rsync-out.txt

-h
Human-readable output – I want to see how many GB and MB were copied/deleted, not bytes.. I can`t think that quickly when I`m bored senseless by a backup job

–progress
Show the progress of each transfer – even if extra information slows jobs down, I prefer to see that things are still trundling away

–delete-during
This is the more interesting one.  This will delete files from the TARGET if they don`t exist on the SOURCE.  So if I was to delete some files on the USB drive then run the sync, they`d be deleted from the network drive.  delete-during says “do it during the run, not before the run, and not after the run” – check out the rsync.html link below for the before/during/after settings.

-v -v
Super-duper verbose.  Again, I really like to see things are happening, even if it slows things down.

-rt
-r Copy directories (folders) recursively, i.e. if we see any directories within whatever we`re syncing, copy those as well
-t keep modification times on files; there`s nothing worse than files in the backup having a completely unrelated date/time!

“/cygdrive/u/” “/cygdrive/m/”
The source to copy FROM is /cygdrive/u which is the top level of the U: drive
The destination to copy TO is /cygdrive/m which is the top level of the M: drive

In other words, this job will copy everything from U: to M: or from the USB drive to the network drive.  This is the catch-all backup-all script that I`ll run once in a while to make sure any other backup batch files I create don`t miss files.

Something a little more useful, and faster:

Backing up a specific directory (folder) backupphotos.bat

So, I don`t necessarily want to backup everything, but instead I want to backup something specific.  Perhaps my super-sexy photos:

“C:\Program Files\Synametrics Technologies\DeltaCopy\rsync.exe” –log-file=”/cygdrive/c/rsync-out.txt” -h –progress –delete-during -v -v -rt “/cygdrive/u/photos” “/cygdrive/m”

This runs a lot faster for me as it will only sync the photos directory to the network drive.  If I only wanted to backup the “christmas2009″ directory, I could run the following, which will be faster again:

“C:\Program Files\Synametrics Technologies\DeltaCopy\rsync.exe” –log-file=”/cygdrive/c/rsync-out.txt” -h –progress –delete-during -v -v -rt “/cygdrive/u/photos/christmas2009″ “/cygdrive/m/photos/”

Obviously it would be possible to create a batch (.bat) file for every source directory on the USB drive, but that wouldn`t make a lot of sense; it would take too long to keep those batch files up-to-date, and they`d more than likely not be generic enough to be very useful.  Instead, although taking longer to execute, I have 3 or 4 batch files, 1 to rsync absolutely everything (on the off chance), 1 to rsync photos, 1 to rsync work files, and 1 batch file to rsync just media files (or more specifically the M:\media directory, specified as /cygdrive/m/media).  Each batch file is identical to those above, just with a slightly different source and target directory.  If I know I`ve changed a lot of photos, I`ll run the batch file to rsync the photos, same with the work files directory.  Takes a lot less time and keeps me happy that I have an up-to-date backup.

The above is not using an rsync server, which means you lose one of rsync`s selling points, in that it can work out if only sections of files have changed, and send only those changes; instead, rsync will see there`s been a changed file and re-copy the whole file to the target.  It however is a more simple way to get the rsync working without fiddling around with DeltaCopy apart from installation.

You can see all sorts of other rsync goodies on this page: http://www.samba.org/ftp/rsync/rsync.html If you`re playing around with rsync, I`d recommend not using a “delete” or “prune” flag.  Mmmm prunes.

In other uninteresting news, I`m down with a nasty bug that`s kept me off work for 2 days.  Not nice.

A few weeks back, I updated the iPhone software to version 3.1, a minor release apparently, and since then I`ve had little but random trouble with the beast.

It all started with the update, which failed, resulting in a bricked phone, and 4 attempts at a restore until the device came back to life. I applied the restore, and hoped all would be back to pre-3.1, but alas no. I`d lost all my apps, some settings, and was faced with the boring task of setting it all back up. I had to dig through e-mails from Apple listing everything I`d downloaded, manually download one-by-one all the apps, and re-set calendar syncing. I`d re-installed the laptop at this point, so the downloads weren`t just there to drag across. In the app store, some apps I had paid money for no longer existed. I contacted Apple as it became apparent that this was a little bit odd, and got nowhere apart from “that app is no longer available”. Thankfully I found an old copy of the download on a backup drive, and I`m very glad the app cost a quid and not ten pounds. Bad show, Apple.

The issue I`m now having is that every once in a while, when the phone is in standby mode, it just won`t wake up. It requires a full reboot, and as I`ve said before, this takes a pretty long time. With no progress meter. The longer I have the phone, the more shoddy the software seems.

It seems a lot of people have had similar problems, with the freezing, particularly on the iPhone 3G: http://www.theiphoneblog.com/2009/09/16/iphone-31-bugs-random-freeze-shut/
so roll on 2010 when my contract with O2 is up. I`d rather have a featureless but sturdy as a brick old school Nokia than a flashy nice but flakey as dandruff device.

Berry Nice

Originally uploaded by reckless_rik

A little photo to spice up the UK autumn. Unfortunately I`ll be working weird hours until the end of October, so will probably miss most of the good stuff out and about. Hopefully, in November, I`ll be able to go mad with the camera and take 1,000,000 photos. They should be uploaded sometime in 2014.