Costs of Data Retention

Does anyone remember the good old days, when developers were called "programmers" and used to pride themselves on the quality and brevity of their code? There's a distinct dark side to capacity, speed and data being increasingly free at any small increment; the cost is invisible to the individual programmer, so its disregarded. We all know about bloatware, and its impact on the speed of your computer. Less reported, but increasingly clear to me are the costs of network traffic. The internet, your LAN and your phone network is faster all the time, but its not unlimited, especially as transaction counts rise due to more users, more devices per user and more bloated software wanting to know more about you. But what about the cost of data retention? Now that space, even on the smallest device, seems free, too much is retained, and far too much information that should be considered secret. Yes, there's retries, preserving for the session, etc., but as a result of this, data is being retained forever. An ID badge I use every day has the SSN sitting there in plaintext. Why? Okay, so isn't that secret anymore. How about my credit card? The industry has some solid metrics and standards that were supposed to be implemented in 2005. How are they doing? Visa has been running a series of polls. Remember, this is self-reported, not an audit by Visa. It could be much worse than this. eWeek: Retailers not exactly where visa wants them to be

But given that Visa has said that there are 1,057 retailers in that group (327 Level 1 U.S. retailers and 730 Level 2 retailers), that four percent suggests that about 42 major retail chains aren't even claiming that they've stopped retaining that data. Visa estimates that the 96 percent relates roughly equally to both groups, suggesting about 13 retailers in the Level 1 group (with the very largest retailers) and about 29 in the Level 2 group.

This is mostly things like retailers retaining the entire contents of the magstripe, forever. I can only imagine why, and I'm hoping the more I write the more I get towards and understanding of how good requirements become bad specifications, and business rules just get lost.

Ha! CC numbers can be linked to issuer!

Back when I started with this whole web design thing I was able to find some scripts to take advantage of some of the check encoding built into credit card numbers. I think we even had a little checksum so we could tell if it was a validly formatted number, but most of all there was a system to the numbering, so we could tell issuer networks apart.

We had to provide this tag for whether the card is Visa, MC, AMEX or whatever to the processing bank, but we could determine that based on looking at the number. I knew this to be true from earlier in my life, so went looking and it was great.

Now, a few years after I get to Sprint I am told this no longer works. Though no one can show me proof, and I don't believe it fully, I'm told that due to the large number of cards issued, that scheme doesn't work so we'll need to ask the customer which card they are using. Its always lame to provide an extra step, but everyone else seems to have done it, so who am I to argue.

Well, its demonstrably not true:


I just recently bought something thru this store with a PayPal checkout scheme. On the left, you see they accept the usual assortment of cards. On the right, after entering my valid card number (that is not it!) it shows which one you have. No user input required. Neat.

I suppose its possible they are using Ajax to actually pre-process the card number, but for a bunch of reasons, I doubt that. I think its parsing the CC number itself on the client side.



Some other time I'll rant about how amazingly poorly the CVV2 code is implemented from a design, usability, comprehension and mostly security point of view. Note that mine was auto-filled by the browser. That's secure.