Publishers are clearly reluctant to allow AI training bots, but it's imperative they allow access for AI crawlers.