Distributed data storage systems

I'm looking for a distributed data storage system. Ideally such a system would have some (or ideally all) of the following characteristics:

• Software only, using shared-nothing (must)
  
• Stores arbitrarily large (actually 2G would be enough) items of binary data accessed by a key (a short-ish string would do) specified at store-time. Items would be stored across multiple storage nodes.
  
• No single point of failure (preferable, a single point of failure which does not immediately impact the service would be acceptable)
• Keeps N copies of each item in different nodes, specifyable either in config or at store-time
• Automatic repairer to re-duplicate items following a storage node's demise (or administrative removal)
• Automatic expiry of old data after a time specified at store-time
• Managability: all nodes would share the same config; nodes can be administratively added and removed without any explicit config changes to other nodes.
• Storage management: nodes should be able to be configured to use a given amount of maximum space; nodes should be able to be put into "readonly" mode where new data are not accepted
• Automatic balancing of load for storage of new items
  
• Monitoring: some cluster-aware monitoring tools which could report on the number of available nodes, total space available, which nodes were almost full, how much data is broken and being repaired, etc.
  

This is a bit of a tall order. There are some systems which almost do it, or do many of the above.

Tahoe seems to be the closest so far.

Of course things like Amazon S3 must do at least most of the above internally, but they aren't open source, indeed you can't even buy it except as a service.

REPAIR TABLE does not always upgrade it

MySQL 5.0 uses a new MyISAM table format than 4.1. So when upgrading, the tables need to be upgraded.

This can be done using REPAIR TABLE. Or can it?

• On 5.0.67 (I've been testing with), REPAIR TABLE does indeed upgrade it
• On 5.0.22 (Earlier version, used on a backup slave), REPAIR TABLE does NOT upgrade it

Oh dear. That breaks my plan for how to upgrade the slave. Not a disaster, but annoying nonetheless

NB: this is a non-production system, and this is the kind of thing that we discover by testing it.