Test downgrade

[engelsanchez]

The 1.7.0 version now has different file formats:

• Hint files now use a bit in the size field to signal a tombstone. To old code, it would look like a massive file offset and therefore a corrupted hint file. Users would likely have to delete hint files in case of a downgrade.
• Data files now contain 2 new tombstone formats. To Riak, they would look like invalid objects, which would hopefully always end up as a not found.

We need to verify that there is a path to downgrade, however annoying.

/cc @jonmeredith

[jonmeredith]

Assigned to 2.0 final - thanks Engel.

[engelsanchez]

Well well. The result of my testing was unexpected: Downgrade is not possible, but it has nothing to do with the tombstone changes mentioned here. The work done to support a more efficient form of key encoding makes the new keys unreadable by the older code. I believe related to this commit: basho/riak_kv@2340346#diff-709ab916aa28dc2a7f28acd8874529dd

I'm unable to read values whether they were deleted in 2.0 or not. Also, when listing keys the workers crash trying to b2t the keys:

2014-08-27 18:27:01.830 [error] <0.608.0> CRASH REPORT Process <0.608.0> with 0 neighbours
 exited with reason: bad argument in call to erlang:binary_to_term(<<2,0,1,98,49,56>>) in 
riak_kv_bitcask_backend:'-fold_keys_fun/2-fun-1-'/4 line 461 in gen_server:terminate/6 line 747

So, as we stand now, there is no easy downgrade path without some kind of tool to reformat all the data :(

[engelsanchez]

Although I laughed at this option once, maybe we should do a release in the 1.4 series that can read this new format. I have now seen that approach taken with other NoSQL DBs, although it's less than optimal.

[evanmcc]

Let me know if there's anything that I can do to do help.

It's also possible to just not use that code, I think just with a config
change, or an easy code change.

On Wed, Aug 27, 2014 at 3:44 PM, Engel A. Sanchez notifications@github.com
wrote:

Although I laughed at this option once, maybe we should do a release in
the 1.4 series that can read this new format. I have now seen that approach
taken with other NoSQL DBs, although it's less than optimal.

—
Reply to this email directly or view it on GitHub
#178 (comment).

[engelsanchez]

2.0.0 released with this downgrade problem. A script is available for people who need to downgrade back to an old version after moving to 2.0.0.