download free 30 days trial version buy Bucket Explorer

Upload of large file fails

On-line support for Bucket Explorer for Amazon S3

Upload of large file fails

Postby Martin » Sun Nov 18, 2007 12:17 pm

Bucket Explorer is a nice tool and really evolves with every update. Congratulations.

I seem to have come across a reproducible bug: A rather large file (543 MB) is uploaded until the progress bar reaches the 100% mark. At this point the file does not appear in the object list of the bucket, instead Bucket Explorer begins to upload the same file again. This process repeats itself until I click "Cancel".

Below an excerpt from the log file.

Thanks,
Martin

Info log:

17 Nov 2007 19:14:08,399 INFO bm Info : Calculating hash of the file :-LS - 2007-10-31.pst
17 Nov 2007 19:16:03,264 INFO bm Info : Calculated hash of the file :-LS - 2007-10-31.pst
17 Nov 2007 19:16:22,241 INFO bm Info : Uploading in progress
17 Nov 2007 20:53:31,984 INFO bm Info : Disconnected
17 Nov 2007 20:54:02,007 INFO bm Info : Reconnecting...
17 Nov 2007 20:54:02,278 INFO bm Info : Connected...
17 Nov 2007 22:47:16,567 INFO bm Info : Disconnected
17 Nov 2007 22:47:46,571 INFO bm Info : Reconnecting...
17 Nov 2007 22:47:46,831 INFO bm Info : Connected...
18 Nov 2007 00:21:03,779 INFO bm Info : Uploading canceled...Archive/EMAIL/LS - 2007-10-31.pst
18 Nov 2007 00:21:03,899 INFO bm Info : Uploaded-2 Not Uploaded(Identical)-0 Skipped-0 Failed-1

The cancel event above occured when Bucket Explorer had failed to add the file to the object listing and started to upload it again. At this point I interrupted the process.
Martin
 

Postby saurabh » Sun Nov 18, 2007 3:39 pm

Martin,

Thanks for reporting this.

If you don't see the file on S3 after successful upload, that means there was a problem during the upload. We calculate a hash of the file before we start uploading the file. We send this hash value to S3 and S3 also calculates the hash after the upload is completed. If the hash provided by us does not match with the one calculated by S3, the file is not stored. So, that tells me the file was not uploaded 100%.

The disconnets and reconnets shown in the log show that there was a problem with the net connection. Are you connected on a slow network / wireless?

This is the 4th report of such behaviour in current production release with large files. So far we have considered these as problem with internet connection, but now we will do a complete review of code again and see if we can solve this.

Thanks again for the detailed logs.

Saurabh
saurabh
 
Posts: 159
Joined: Tue Aug 26, 2008 8:30 am


Return to Bucket Explorer for Amazon S3 - Support Forum