6 years later, IT projects still can't cope with change

I read in Ars Technica that Most IT projects in Europe are late. According to the article,

The numbers listed in the original BBC article, suggest that the European country more likely to deliver on time is Sweden, where 44% percent of IT projects make or time. Spain and Russia are at the bottom of the list, with only 4% (!!) of IT projects delivered on time.

This is June 2007, more than 6 years after the Agile Manifesto was published. And we still can’t manage the volatile nature of IT requirements. Apparently we can’t really manage much in IT, anyway.

I wonder why is that? Is it really all that difficult to manage customer expectations and work in nearly-continuous-re-planning mode, adapting to changing priorities and requirements?

There are successful IT companies that hardly ever need to push a deadline, so it is definitely possible to do a good job of an IT project and to do it on time, too. So is really the majority of the IT industry so incompetent?