Minimal 4.0 COMPACT STORAGE backport

There are several behavioural changes related to compact storage, and these differences are larger than most of us have anticipated: we first thought there’ll be that “appearing column”, but there’s implicit nulls in clusterings thing, and row vs column deletion.

Some of the recent issues on the subject are: CASSANDRA-16048, which allows to ignore these differences. The other one was trying to improve user experience of anyone still using compact storage: CASSANDRA-15811.

Easily reproducible differernces are:

(1) hidden columns show up, which breaks SELECT * queries
(2) DELETE v and UPDATE v WITH TTL would result into row removals in non-dense compact tables (CASSANDRA-16069)
(3) INSERT allows skipping clusterings, which are filled with nulls by default.

Some of these are tricky to support, as 15811 has shown. Anyone on OSS side who might want to upgrade to 4.0 while still using compact storage might be affected by being forced into one of these behaviours.

Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these.

It looks like it is possible to leave some of the functionality related to DENSE flag and allow it to be present in 4.0, but only for these three (and potential related, however not direrclty visible) cases.

e.dimitrova since you were working on removal on compact storage, wanted to reassure that this is not a revert of your patch. On contrary: your patch was instrumental in identifying the right places.

cc slebresne aleksey benedict marcuse