I think that it should be possible to test your business logic outside of any container but the jvm, yes. So all those big ugly session beans (now don't get me wrong, I like session beans, just not these ones) bloated with fat service logic and business methods ought be be testable without an EJB container. Or, in fact, the entire session bean. When I want to know when my company identity matching logic functions as specified, or credit approval heuristic on known inputs produces the right results, and so on, I don't want a big ugly chunk of infrastructure in my way.
Similarly if I'm using Hibernate for persistence I can also check persistence objects and mock objects to test the DAOs. Outside of having to start up a thing like a J2EE engine like Websphere or Weblogic and installing the beans into that. Much easier to use as little as possible to verify something with a paucity of extra engines. You have to start the JVM anyway, which is not optional in any other case, so they become JVM + Websphere + DB2 + Hibernate + and so on. Let's just stop at the JVM if we can possibly do it. Less interactions to pursue in case of nasty bug..err unintentional features!
I.e. unit tests. No test container please.
But when it comes time to try out if a remote client can invoke the logic behind a service delegate or session facade, that the container will handle the dependencies, that the transactional scope and contexts are correct and exceptions handled sensibly, etc, then sure, I want to test the whole container dodaddet whigjammery box and dice caboodle. Both in pieces and in a complete integration test.
In short, I think the answer is, know what it is your are trying to test at any point. Or at least that's the question if not the answer. I don't want to be testing if findMeASucker(whatsit, honeydew) logic works according to requirements. I want to test if findMeA* methods function in a sensible way to the way the client delegate or whathaveyou expects it to work. In normal situations and boundary conditions.