Did the Crusaders destroy the Byzantine Empire?

Justin asks how crippling the Fourth Crusade was for the Byzantine Empire.

Many historians view it (with good reason) as having destroyed the empire- or at least started it on its final decline.  The most obvious impact was economic.  During the initial three days of plundering a vast amount of loot was seized- enough for the Crusaders to breathlessly report that they had captured one fourth of all the wealth in the world.  The pillaging, however, didn’t stop with the initial assault.  For the duration of the Latin kingdom of Constantinople  (roughly sixty years) relics and artwork continued to stream out of the city as hidden caches were found and houses, palaces, and churches were stripped bare.  The more prestigious articles ended up in the reliquaries of western Europe- many in cathedrals established specifically to accommodate them. (Notre Dame was built to house the head of John the Baptist)  The exodus of wealth left Constantinople impoverished and by the end of the occupation, the Latin emperors were reduced to stripping the lead off of the roofs of the imperial palaces.

When Michael VIII recovered the city in 1261 he found it in a sad, dilapidated condition- ‘shrunken like an old man in the clothes of his youth’.  Deserted houses were still sagging from the fires that had damaged them more than half a century before, and vast stretches of land were given over to weeds.  Even worse than the physical destruction was the psychological damage.  The unity of the empire was permanently shattered.  Where there had been one Orthodox Empire there were now three successor states- Nicaea, Trebizond, and Epirus- all claiming to be the same thing.  Epirus eventually succumbed to the restored power of Constantinople but Trebizond never did.  In the view of most Byzantines the relationship with the West- both moral and diplomatic- had been irrevocably sundered.  When the final end came, they would famously say ‘better the Sultan’s turban than the Pope’s mitre.’

But despite the tremendous physical and mental impact of the Fourth Crusade, is it really fair to say that it destroyed the Byzantine Empire?  In the years leading up to the Crusade Byzantium was decaying from the inside, plagued by foolish leaders, aristocratic greed, and military incompetence.  It was a very sick patient by 1204 and its illness went much deeper than a weak army or neglected defenses.  It was dying a slow death before the first Crusader showed up, and it lingered on for another 249 years after the sack.  Did the Fourth Crusade weaken the empire? Certainly.  Did it hasten its demise? Probably.  But did it destroy Byzantium? No.