Is the room completely open or is there a 'doorway' that they need to pass through?

You could put an invisible npc @ the doorway and set it's proximity to be a line / ellipse rather than a circle.

If the room is an open-air room, you could bound the area by 4 invisible npcs, setting their proximities to create a square/rectangle around the room.

What about a hailable npc to get in, flagging the hailer, and using the boundaries set by invisible npcs?

If there is a doorway you could also use a player.pl with the sub EVENT_CLICKDOOR.

Just have it increment a counter ++1 for each click with a quest::movepc and if counter > 6 it gives them a message saying the encounter is full? I haven't dove into global variables but you could have the counter tied to one of them and then use a sub EVENT_SLAY on the npc in the room to decrement the counter if he murders someone.

If no door I'd use the proximities & 2 different inaccessible areas of a zone....add a crate to a hall or something to block it off.