Crystal can be destroyed only with fire, but fire is meant as a spell with area fire effect. No fire weapon, only area fire based spell. If you have a mage in your group, try to use some kind of area fire based spell and you´ll see everything will continue seamlessly...
And I bet you can found a wand of fire somewhere in the tower. But now it´s too late to come back...
EDIT: ...already answered by yourself... About the book - in V2 it is just a cosmetic option. Wait for V3, it will have some meaning in later stage of game.