why it this separate mechanism needed in the first place?

Because ActivityPub was not designed for E2EE. That’s the simplest answer.

The longer, and more technical answer, is that doing the actual “Encryption” part of E2EE is relatively easy. Key management is much harder.

I initially set out to just do E2EE in 2022, but got roadblocked by the more difficult problem of “which public key does the client trust?”.

It’s a building block to make E2EE possible at Fediverse scale.

I’ve written about this topic pretty extensively: https://soatok.blog/category/technology/open-source/fediverse-e2ee-project/

If you can build in Federated Key Transparency, it’s much easier to reason about “how do I know this public key actually belongs to my friend?” which in turn makes it much easier to get people onboarded with E2EE without major risks.