Data privacy is without a doubt one of the defining challenges of the digital age. For too long, the issue has been wrapped up in clichés around Big Brother and the surveillance society that provoke much argument but no proper debate.

The government, it seems, is only too willing to allow others to shout at each other and avoid any meaningful solutions while it continues its database-creating frenzy.

The regularity of stories revealing further areas where Whitehall’s creeping influence has grown over people’s information only serves to foster the image of a government that wants to use our data how it can, while it can. Our story this week that central government workers will have to hand over their bank details – ­ joining their counterparts in local authorities – ­ is another example.

The government has genuine reasons to want all this information, and it is easy to build a justification around crime reduction, better public services and lower costs. Yet too often it shoots itself in the foot by losing data or over-reaching itself.

The European Court of Human Rights last week put the national DNA database firmly into the latter category, ruling that two British men who were arrested but not convicted of crimes should have their records removed from the system, stating that the UK government “had overstepped any acceptable margin”.

Next year a new Information Commissioner will be appointed, and he or she will take on the role in a very different environment from the one in which the present holder, Richard Thomas, started. More than ever, this increasingly high-profile position needs to be held by someone who will champion the privacy of the individual.

The only long-term solution is to give back control of data to the people who own it ­ – and that is us. Emerging privacy-enhancing technologies will help to make this happen.

With that firmly in mind, it is time for the government to lead the privacy debate, to make itself a beacon for responsible use of our information, and not to grab as much data as it can before more people resort to the law to prevent it.

For those unfamiliar with the UK and its pleasant pastures seen, the changes being introduced this week will pass unnoticed. Indeed, for the vast majority of Britons, the fact that foreign nationals who apply for residency will now be required to apply for identity cards may also pass unseen. But this is an important change, and one with which no Briton can be entirely comfortable.

For too long, this government has been wed to the idea of introducing identity cards, without elucidating a clear argument for so doing.

This paper has consistently argued that some form of standardised electronic personal identity management system would be invaluable to citizens, and their desire to safeguard online and physical transactions, without compromising their rights to privacy.

Yet the ham-fisted attempts to first impose an ID card on a sceptical public, followed by the latest ruse to sneak it through the back door, have undermined the project. The government’s lamentable record on looking after personal data has heightened public concerns about ID cards.

It could -­ and should -­ have been so different. Consumers are becoming more attuned to the importance of identity in this digital world. The technologies that may help protect individuals and provide a robust identity, such as biometrics, are gaining acceptance.

The use of biometric readers, such as fingerprint scanners, even iris readers, are no longer the preserve of James Bond movies. They have become a workaday part of many of our lives.

As is so often the case with government IT projects that go awry, it is not the technology that is to blame, but the quality of leadership. So even if the deployment of ID cards passes off without a hitch, the scheme is fundamentally flawed because of a lack of public support and the perverse decision to ignore the recommendations of the Treasury-commissioned Crosby study.

It is a shame that for some senior government figures it appears to be more important to dogmatically pursue this cause, rather than take the time to build a system in which we can all believe. It is not yet too late to introduce a workable system for identity management, but it is looking increasingly unlikely that it will happen.

When the anonymous civil servants who drafted the Data Protection Act (DPA) in 1998 set about their work, they were smart enough to realise that the world of computers was likely to evolve considerably.

Keeping the law up to speed with the pace of technological change was always going to be a challenge, and the principles-based approach that underpins the DPA has worked well. Precedents set through case law have allowed the Act to encompass technical developments ­ although Marks & Spencer’s challenge to the Information Commissioner’s enforcement of laptop encryption represents a new test.

But even such legal foresight was unlikely to have taken into consideration the internet, cheap storage, USB drives, broadband, social networking and all the data-intensive applications upon which modern business life depends.

So it is right to heed the calls for a review of the DPA that will see it through its next decade.

But today, law enforcement is not enough to securely protect data. A new DPA needs to go hand in hand with best practice IT management.

When the DPA came into force in March 2000, the article in Computing concentrated on the specifics of the new law and the challenges it posed for IT departments.

Any similar feature now would cover the legal aspects in just a few lines. The complexity of the IT protecting people’s personal information has increased exponentially. And with it, the awareness of identity theft and the potential for misuse of our data has grown in parallel.

Self-regulation and self-discipline are now just as important as legislation.

Every employee needs to be constantly reminded to protect corporate data as if it were their own. And processes such as information lifecycle management need to sit alongside all the security procedures necessary to treat our information with the care we each wish it to be handled.

A review of the law is timely. The review of your IT management practices should be constant.

The oldest and truest cliché about data privacy is that technology is both the problem and the solution.

This is the cleft stick in which the government finds itself when it comes to the use of information in the growing number of databases storing our personal details.

There have already been murmurings that ministers want to relax elements of the Data Protection Act to allow further cross-matching of data in different Whitehall systems.

Where such a practice has been put in place, there have been successes. For example, cross-referencing between visa applications and the police fingerprint database has led to the arrest overseas of people that committed crimes in the UK.

Perhaps the highest-profile example is the car tax disc web site, which combines information on MOT results and motor insurance details to deliver one of the better online public services.

But the downside of this cross-matching has privacy campaigners up in arms.

What might the government learn about each of us were it to piece together fragments of our lives scattered across disparate departmental databases?

The fundamental principle of data protection legislation is that personal data should only be used for the purpose for which it was originally recorded. Excessive cross-matching would be a clear and controversial infringement ­ but you can be sure the government will look for more situations where it can claim that the benefits outweigh the risks.

The difficulty will come from the fact that trust in the way government uses our data is at an all-time low. HM Revenue & Customs has been told in no uncertain terms that it must develop a culture of information security and data protection as the price of the failings that led to the loss of CDs containing 25 million child benefit records.

The rest of government needs to learn the same lesson ­ as do corporations. The potential benefits of information sharing, cross-matching and data mining are real ­ but the risk can only be justified when the right culture and processes are in place.

The realisation is growing that data protection is not somebody else’s responsibility.

Moves to make individuals liable for the loss or disclosure of personal information held by public sector bodies or by companies are an inevitable response to the data loss scandals we have seen in recent months.

Until now, the Data Protection Act has focused responsibility on senior executives of an organisation, who are held to task for failures of staff under their charge. But the reality is that those workers are rarely motivated by the need to protect their bosses or their employer’s reputation.

When someone else takes the blame, why should you care if you make a mistake?

The most common concern of IT leaders looking to introduce data protection or risk management policies is how to create a culture that supports the rules and regulations put in place. Having a policy is one thing ­ making staff buy into it can be quite another. Ultimately, a culture exists only in the collective hearts and minds of a group of individuals, it cannot be imposed from above or through a set of rules.

From a government perspective then, legislation appears to be the only answer.

But no law will be effective ­ other than in increasing the prison population ­ without education to go alongside it.

Data privacy is perhaps the biggest single challenge facing the technology industry. Information security is not the issue ­ technical controls exist to secure the vast volumes of electronic data being generated ­ but the access to, and authorisation of the use of that data is about people, not technology.

For every government employee who inappropriately accesses citizen records (see www.computing.co.uk/2215705), how many would protest if someone did the same to their personal details?

People need to realise the impact on others of their actions, and to be given training and advice to ensure they are aware of their responsibilities. The best way to do that is to ask the question: What if it were you?

Government has been wholeheartedly – ­ if not always successfully ­ – embracing the potential for technology to improve services to citizens, but the politicians have some way to go.

As local councils across the country go to the polls today, few candidates can claim to have turned to the internet to boost their appeal to voters.

The leaflet pushers have been as active as ever ­ – the tree count for all the paper thrown straight into voters’ rubbish bins in the past month must be huge ­ – but this new-fangled computer stuff seems to have passed our potential representatives by.

In London, where the highest-profile election is taking place, none of the candidates looked at the possibility, for example, of using social networking to engage with voters; nor have they made much mention of IT’s potential in improving the citizens’ lot. Ken Livingstone’s jocular anticipation of chips in our heads may not win him another term as mayor, but his Tory rival Boris Johnson can hardly claim to represent the internet generation either.

How different things are in the US, where the battling Democrat candidates, Hillary Clinton and Barack Obama, have put the web at the centre of their campaigns. The forthcoming presidential elections will be the most internet-enabled we have seen.

Perhaps part of the problem lies in the culture of secrecy surrounding technology in parliament.

Only now, after a series of data protection scandals, has the Information Commissioner been given the go-ahead to spot-check Whitehall departments for compliance. And MPs want to improve transparency by gaining access to departments’ management information systems instead of having to wait for annual paper-based reports.

The internet ethos is built on openness ­ – not a quality often associated with politicians. If our elected representatives could grasp how IT can connect them with citizens, not only would they be closer to our tech-enabled culture, but perhaps more open with us all.

It would be easy to be cynical and suggest the government engaged in a bit of press control with the timing of the publication of Sir James Crosby’s report on UK identity management last week.

On the afternoon that home secretary Jacqui Smith announced the latest changes to ID cards, the Treasury-commissioned Crosby study was also quietly released after months of delays ­ – Computing was leaked details of its contents as long ago as last August – ­ see www.computing.co.uk/2197249.

Smith said she was “indebted” to Crosby, but ignored most of his recommendations ­ – not least the widely publicised suggestion that ID cards should be free.

But a detailed look at the Crosby report ­ – which was initiated by Gordon Brown when he was chancellor – ­ reveals a more coherent, workable, and less costly alternative to the increasingly ham-fisted and ever-changing plans for ID cards.

The former HBOS chief executive recommends a system delivered by the private sector through trusted institutions such as banks. The government has co-opted at least part of this, in that companies will be asked to bid to provide biometric enrolment services, but the national identity register remains a Whitehall resource.

Under Crosby, you choose which trusted organisation looks after your biometrics. Far less Big Brother.

His proposal is for a consumer-led process that offers citizens who are increasingly worried about identity theft a secure way to prove who they are, with a commercial incentive for the banks. And of course, public services can piggyback the scheme. Compared to the government’s attempts, it appears to make much more sense.

There is no doubt that in future we will need some form of standardised electronic personal identity management system to safeguard our details and our online ­ – and physical ­ – transactions.

But the government’s lacklustre attempts to sell ID cards to a sceptical public are doing more to threaten this goal than to promote it. The expertise of the private sector needs to be given more weight in the identity management debate.