As recent payment outages in Germany (and elsewhere) have shown, even proponents of a cashless economy have an interest in safeguarding the future of cash, if only for the sake of financial stability.
A few days ago, many shops in Germany had “CASH ONLY” signs displayed on their windows or at the entrance to their premises. Some establishments allowed people without cash to pay by card, but only if they provided a signature. It was as if parts of Germany’s payment system had just travelled through a wormhole back to the 1990’s, albeit with euros rather than the Deutsche Mark as the legal currency.
According to initial reports, the cause of the problems was a software glitch affecting all H5000 payment card terminals, which are widely used by German retailers including some of the largest supermarket chains. Starting on May 24 normal card payments became all but impossible for those retailers. The problems lasted for a week or so, prompting some larger retailers to replace all of their card terminals. It’s an investment that many smaller, struggling retailers can ill afford to make right now.
There are (or at least were) around 100,000 H5000 units in Germany, according to some estimates. The devices were manufactured by the US financial services provider Verifone specifically for the German market, but they are operated by eleven network operators, including Payone GmbH and Concardis GmbH.
On May 27, one of those companies, Payone, reported it was facing issues with the H5000 card terminal and that the issues were occurring throughout the country: “Like other network operators, we are currently experiencing considerable restrictions in the processing of transactions with card payment terminals of the type H5000 from the manufacturer Verifone throughout Germany.”
According to Handelsblatt, financial supervisors from Bafin and Bundesbank are already on the case. The German banking industry, which represents the interests of banks and savings banks, also announced that it would “analyze and process the disruption in depth in collaboration with the various parties involved and the supervisory authorities”. However, it also conceded that it will still take “a while” before the last H5000 terminal is replaced or updated.
The problem first came to light on May 24 when the Konsum retail chain in Dresden published the following message on its Facebook page: “Attention, an important notice for you! Due to a Germany-wide malfunction, card payments are currently not possible in our stores.”
Similar problems began to be reported by other retail chains such as Netto, Edeka, Aldi Nord, Rossman and DM as well as smaller, independent retailers and petrol stations. Initial reports suggested that H5000 card machines across Germany were experiencing a software malfunction that stopped them processing payments.
“As things stand, it will be necessary to install new software updates on all H5000 terminals, which the manufacturer will provide as soon as possible,” Payone said.
But some IT experts have pointed to a different possible cause: an expiring security certificate. The problem, it seems, is that the Verifone H5000 is a rather old albeit robust model whose so-called “End of Life” was officially announced by Verifone in 2019. The company ended its production of the terminal a year later. Although it offered limited product support until 2023, one of the embedded certificates seem to have expired unnoticed on Tuesday May 24 causing the terminals to stop working. That is the hypothesis of Jan Wildeboer, a self-described “EMEA Evengelist” at IBM-owned Red Hat who began informing the English-speaking public about the issue in a rapidly expanding Twitter thread.
Wuildeboer is a Linux expert and the H5000 was the first card terminal in Germany to run on open source software. On May 30, the Bavarian newspaper Süddeutsche Zeitung reported that a “curious detail” within the network operators’ support instructions seemed to support Wildeboer’s hypothesis (machine translated):
Usually, any tech support will immediately ask the caller, “Have you tried rebooting?” In this case, the network operators advise retailers against rebooting the terminals themselves. That fits with Wildeboer’s certification thesis. Because when you reboot, the devices check their integrity, i.e. whether they may have been manipulated by hackers. If they then lack a necessary certificate, they switch to a kind of safe mode from which only an on-the-spot technician can release them.
Who’s to Blame?
Trying to apportion blame for the payment terminal outages is not going to be easy. As the article in Süddeutsche Zeitung notes, Verifone released a software update in Verifone despite having previously stated there would be no more updates after April 2021. This suggests the developers may have realized there was a potential problem with the expiring certificate. But the update seems to have reached only a few retailers, who apparently experienced no problems with their terminals. For everyone else, the devices suddenly became unusable on May 24.
This, as the article notes, is where the question of guilt gets interesting. After all, retailers in Germany have clearly lost a certain amount of business as a result of the outage. If Verifone made a software update to avert the problem, then who is responsible for the broken devices? The retailer? Or the network operator? Or did Verifone fail to give the network operators and retailers sufficient warning about how critical the unscheduled update was?
Inherent Fragility
In recent days the crisis appears to have more or less petered out, at least for the companies that have been able to replace or fix their terminals. On June 3, the newspaper Focus reported that more and more of the affected retailers can now accept credit cards and giro cards again. But the crisis still serves as a timely reminder of one of the main dangers of going completely cashless: the inherent fragility of a purely cashless economy.
At a personal level the impact of the payment outage in Germany was fairly limited, as long as you had access to cash. And just about everybody did, given that ATMs continued to work throughout the outage. In other words, there was an alternative to digital money that was widely available to all users — i.e. physical cash.
There is also something rather peculiar about Germans (as well as Austrians): they love physical money (a topic I have written on before for WOLF STREET). Even the intense demonization of cash in the early days of the COVID-19 pandemic failed to shake the German people’s faith in physical lucre, although the use of cashless payment transactions, whether by card or mobile, has grown significantly in the last two and a half years.
A recent 11-country survey by the payment companies PYMNTS and Stripe found cash and cheques are still the most common way to make in-store payments in Germany, accounting for 37% of all in-store transactions compared to 29% for card-based methods and 20% for mobile payment platforms. The study — Benchmarking the World’s Digital Transformation — found that cash is also still popular in Italy, Spain and Japan.
In many other parts of Europe, particularly in the north, cash is, to all intents and purposes, dying a natural, albeit artificially accelerated, death. For example, in France cash and cheques account for just 12.6% of in-store transactions. In the Netherlands it’s 18% and in the UK, 19%.
A Brief Foretaste
Just imagine what would happen if a similar payment card outage that just happened in Germany were to occur in Sweden, Norway, the UK or France, where far fewer people use cash. The disruptive impact would have been far greater. We already had a brief foretaste of what that might look like a few weeks ago. On Monday May 16, Norway, the world’s most cashless economy (according to its central banks), with reportedly just 4% of the population still using cash, suffered a nationwide outage of its card payment system. Card terminals went down throughout the country, leading to long queues at the ATMs.
“In a large number of shops, including grocery stores…, payment cards were useless on Monday afternoon,” reported the online newspaper Nettavisen. “Many feared that the ATMs would run out of cash as a result of the unexpected development.”
Imagine if the outage had lasted for days or weeks rather than hours, as occurred in Puerto Rico in 2017 as a result of devastation caused by Hurricane Maria. Electronic payment systems went down for weeks, turning the island into a de facto cash-only economy. So serious was the fallout that corporate clients began begging the New York Federal Reserve for large amounts of dollars in cash to meet payrolls. In the end, the Fed dispatched a jet loaded with an undisclosed amount of dollar bills to the stricken island.
Of course, there are myriad other reasons why a fully cashless society is far from desirable, including the inevitability of more granular surveillance, the loss of one of our last vestiges of personal privacy and anonymity, the exclusionary effects it will have on those who are unable to access or use digital technologies and the much greater power and control it would grant to both governments and corporations over our spending habits — and indeed potentially over our ability to spend money at all.
As Silkie Carlo, the director of the British civil liberties organization Big Brother Watch wrote in a recent op-ed for the Daily Telegraph, financial services companies have already shown a predilection for taking an interventionist approach to people’s spending:
There are now numerous examples of cancel culture driving digital wallets to be frozen – an early notable example was PayPal freezing WikiLeaks’ account in 2010. Just months ago, the Canadian government froze bank accounts of people associated with the truckers’ “Freedom Convoy”, in an effort to quell the campaign against mandatory vaccines. With digital currency, the question fast becomes not only who watches how you spend your money, nor even who controls how you spend it, but who actually owns it?
Even people who have absolutely no qualms whatsoever about the prospect of governments, central banks and corporations instituting a cashless economy as a means of grabbing a lot more power, or arguably total power, over our lives should have serious concerns about its potential implications for the stability of the financial system. As Stefan Ingves, the governor of Sweden’s central bank, the Riksbank, warned back in 2018, replacing physical cash altogether could cause serious problems in times of crisis:
“If the power supply is cut it’s no longer possible to make electronic payments. For reasons based purely in preparedness we need notes and coins that work without electricity.”
Of course, central banks themselves are exploring the possibility of introducing their own central bank digital currencies, or CBDCs, in the near future. A small number have already done so, including the People’s Bank of China, whose digital yuan is already being used in pilot schemes to stimulate consumption in its pandemic-hit economy. While it’s true that many central banks have insisted that physical cash will continue to exist alongside the CBDCs, central banks are not exactly known for keeping their word.
17 comments
Comments are closed.
Observation from Germany: It was a holiday weekend and our small town gets a lot of visitors. Only cash at the supermarket – and by midday on the Friday the town’s four cashpoint machines were all empty too.
Great post. Thanks very much.
To repeat an old anecdote, we had massive floods where I live back in 2011 with problems being experienced in not only electricity but also the phone/internet lines. Fortunately I had cash at home which could help out but even then, some places closed their doors because they needed telecommunications to run their businesses which was crazy when you think about it. So if you lived in a country that abandoned cold cash and relied entirely on digital cash, there is only one thing that you could do. Have a supply of goods that would work in a barter economy. So if electricity was the issue, candles, matches & lighters might be worth having a supply off to barter with. Yeah, if your digital economy falls down, we are back to a barter economy again. And remember that with climate change, that problems with electricity and phone/internet will be more frequent.
And this while all the hard sciences are warning that crazy weather will just get worse thanks to the fossil fuel driven warning.
So expect more, and bigger, floods, more thunderstorms playing havoc with anything electric, etc.
And one part of this is down to the callousness of business and public management.
Norway had a hour long outage of their payment system just a few days before Germany, and the stores made do with anything from cash only, people using a service called Vipps to transfer money between themselves and storage management, and in various rural places simply signing an IOU with the store as everyone knew each other on a first name basis anyways.
Still, the Norwegian coastline get hit by some severe storms each winter. And one that is still remembered hit in 1991. It left a large stretch of the coast without power for days. But landlines still operated, as long as the wires were intact, thanks to the then public phone company having people driving around with generators to top off batteries.
But a more recent, and far from as damaging, storm had the phone networks, landline as well as mobile, be down for most of a day or two because the now privatized networks could not care less. After all they had a requirement contract with the government and stuck to it like glue.
And as best i can tell, all these new payment systems etc seem to come with a similar shift towards blaming the customer for carelessness. This even as management lands fat bonuses for having cut operational costs by reducing staff, offices and paperwork.
In 2011, on a long weekend vacation to Dan Diego there was a big blackout over 24 hours. Fortunately, we carry cash on trips and bought food at grocery stores using cash. They tabulated totals on little calculators. We cooked on a grill on the balcony and hung out on the beach. Ended up being a fine weekend for us.
Also discovered the grocery stores have to throw everything away that has to be cooled after 24 hours. Lot of wastage.
Could also be an opportunity. In such a situation you could ask if they would sell some of that stock at a discounted price before it goes bad. At least they make some money and all you have to do is cart that food straight home to cook and eat.
When the electricity went out at my store, I promptly broke out the pens, paper and calculators for the staff. We’d note the SKU and price of the purchases, add ‘em up, calculate tax and bam, we’re still in business. We even had the “schick, schick” machines and forms for credit cards.
Electricity? Cash don’t need no stinking’ electricity!
Bravo!
Smart!
However, those card imprint machines are going to be useless soon as the newer chip cards don’t have embossed numbers. Merchants will have to hand write the card details for later processing.
Well, it looks like just another reason to be a Prepper?
“Prepper” as in to be prepared for something? I have generally enough canned and dry food on hand for 1-2 weeks in case I can’t get to the grocery store for whatever reason. Am I a “prepper”?
I also use cash at the grocery store, and have several anecdotes (I’ll spare you) where the cc or db card readers went down, checkout lines getting longer and longer, leading store manager to step out and say ‘yada yada credit terminals down, just wait’ and a few people then piping up with a “what about cash” question. Said manager taking a minute (seemingly for a moment to take in the question about cash payments) to respond “oh, cash payments can be made in line 2” (or whichever line). Followed quickly by many people waiting in line moving their grocery carts to line “2”. / ;)
adding: living in the analog world. / ;)
Next week the Oregon coast will enjoy the “Cascadia Rising” exercise, in which the authorities will attempt to practice their response to a massive tsunami. Part of the exercise involves distributing literature encouraging people to prepare a “Home 96-Hour Kit”. This must include water, food, blankets, essential medication, a first aid kit, a fire extinguisher, and… money.
Like the Rev, I’m not sure how much use that money will be in the event that the coastal bridges have all collapsed, beachfront homes been destroyed and no electric power or food delivery. I suspect we’ll be bartering the contents of our 96-Hour Kits, or – imagine! – sharing them with our neighbors.
Random comments:
Money is the pointy tip of the workings of any economy, but the whole depends on information flow: incredible coordination is needed between otherwise loosely coupled actors to make and move our goods and provide our services. As Ken Murphy and Ghost in the Machine point out, a fallback to paper including the “schick, schick” machines is possible in the short run to handle payment problems. Tracking inventory, matching it to sales rates to identify re-order amounts, contacting vendors to re-order, being able to trust them to fill the orders, all this is a slog, muddy ground impossible to get traction in, if communication fails generally, due to power failures or because of natural or unnatural events.
Sometimes I worry about the fragility of our whole economic system at the plumbing level, even though most of the discussions here at NC are at the economic or social decisions level.
—
Physical cash is consensus money. Money in bank accounts is backed by the promises of billions of people both as individuals and via their associations – businesses, corporations, governments, and so on. Consensus money has the value that people agree that it has.
Several actual and historical characteristics allow consensus currencies to keep value; it cannot easily be counterfeited; it is accepted for payments by the government for things like taxes; there is a history of it being used as money; during that history it has kept value.
Consider, for example, the afghani. With the collapse of the Afghan banking system, it has become disconnected from promises, so has become a consensus currency. Since the Afghan economy is damaged and the supply of goods and services is reduced, you would normally expect rising prices. On the other hand, since the supply of afghanis is limited, their value as a tool of commerce is greater, so you would not expect hyperinflation. Furthermore, the Taliban accepts it both for regular and irregular payments (taxes and bribes) on an ongoing basis. The afghani was a reasonably stable currency for many years, and there is still the expectation that someday it will again be a bankable currency.
—
Physical cash is short-term expedient. It cannot replace the more complex economic interactions that are possible in a banking system that tracks mutual promises. Not just banking, but the economy as a whole is dependent on communication. The failure of robust communication implies the failure of society. Having electronic payments fail is a relatively minor warning and an incentive to keep our communication infrastructure strong.
—
I am on team “expiring product certificate” although this is not the most accurate phrase. “Expiring security certificate” is better phrasing, and yes, I’ve both let a security certificate expire because of forgetfulness, and seen ones expire at places I worked, for the same reason.
There is a consistent and understandable story here. Verifone builds a device that it expects to have a particular lifetime. As part of its operation, it uses security certificates, because, you know, security. The certificate has an expiry time, initially chosen to be beyond end of product life. The certificate is hard coded into the device, because, you know, security – you certainly don’t want remote updates of a payment device. Been there, done that, not good.
But the device is popular. Stores don’t want to replace it because it works. They figure upgrades are just another vendor cash grab. The Verifone sales side extends the product life. Forcing a replacement could lead to stores choosing an alternative device, Sales, of course, was not involved with the certificate expiry time decision, and isn’t even aware of it. The technical side has forgotten about the expiration. They built according to specifications and what they built would have had no problems. Unfortunately, specifications changed years after the design and build.
So now any upgrade needs an unscheduled personal visit. Can’t be done quickly. Oops.
Thanks, Ghiggler, for the added detail, as well as the correction of my use of terminology. Have amended the text accordingly.
> . The certificate has an expiry time, initially chosen to be beyond end of product life. The certificate is hard coded into the device, because, you know, security – you certainly don’t want remote updates of a payment device.
Planned obsolecence built right in, good job.
The whole system reminds me of the Internet of Things, albeit somewhat less cheesy.
Thanks for the excellent post.
I would just add that the banks and credit card companies of course cream off a fee every time you make a transaction, which they cannot with cash. At one point in Denmark retailers added the fee to the customer bill for foreign (even EU) cards, but not to Danish cards. I think this practice is now outlawed. I have also selectively seen smaller stores in many countries passing the cost to the customer or, more frequently, giving a discount for cash. So, yes, they will push in the direction of cashless, because they make money on it, track your purchases, sell your data where legal, and when the system breaks down you are the one with the problem!