+1

MD5 is little better than ROT13. At least with ROT13, you have no illusions.

We use SHA 512 for most work. We don't do finance or national security, so it is a good fit for us.

Cary

On Oct 2, 2014, at 12:30 PM, Simon Spero <[log in to unmask]> wrote:

> Intel skylake processors have dedicated sha instructions.
> See: https://software.intel.com/en-us/articles/intel-sha-extensions
> 
> Using a tree hash approach (which is inherently embarrassingly parallel)
> will leave io time dominant. This approach is used by Amazon glacier - see
> http://docs.aws.amazon.com/amazonglacier/latest/dev/checksum-calculations.html
> 
> MD5 is broken, and cannot be used for any security purposes. It cannot be
> used for deduplication if any of the files are in the directories of
> security researchers!
> 
> If security is not a concern then there are many faster hashing algorithms
> that avoid the costs imposed by the need to defend against adversaries.
> See siphash, murmur, cityhash, etc.
> 
> Simon
> On Oct 2, 2014 11:18 AM, "Alex Duryee" <[log in to unmask]> wrote:
> 
>> Despite some of its relative flaws, MD5 is frequently selected over SHA-256
>> in archives as the checksum algorithm of choice. One of the primary factors
>> here is the longer processing time required for SHA-256, though there have
>> been no empirical studies calculating that time difference and its overall
>> impact on checksum generation and verification in a preservation
>> environment.
>> 
>> AVPreserve Consultant Alex Duryee recently ran a series of tests comparing
>> the real time and cpu time used by each algorithm. His newly updated white
>> paper "What Is the Real Impact of SHA-256?" presents the results and comes
>> to some interesting conclusions regarding the actual time difference
>> between the two and what other factors may have a greater impact on your
>> selection decision and file monitoring workflow. The paper can be
>> downloaded for free at
>> 
>> http://www.avpreserve.com/papers-and-presentations/whats-the-real-impact-of-sha-256/
>> .
>> ______________________________________
>> 
>> Alex Duryee
>> *AVPreserve*
>> 350 7th Ave., Suite 1605
>> New York, NY 10001
>> 
>> office: 917-475-9630
>> 
>> http://www.avpreserve.com
>> Facebook.com/AVPreserve <http://facebook.com/AVPreserve>
>> twitter.com/AVPreserve
>>