When a conference is established, the conference is hosted within the frontend pool the meeting organizer is hosted on. This is the case even if there are multiple locations, each with their own frontend pool. In the example where a meeting is established between an organizer in site A who is homed on the Site A frontend pool and 100 participants in Site B who are homed on the Site B frontend pool, separate media streams will be established between each of the 100 participants in Site B and the frontend pool in Site A. With the exception of video (due to different users selecting to view different participants any given time), this design results in an extremely inefficient use of bandwidth for the audio and desktop sharing\application sharing media streams (which should be the same for all users).
A more efficient solution would be for the frontend pool in Site B to participant in the conference and multiplex the media streams between itself and the frontend pool in Site A. All participants in Site B would connect to the frontend pool in Site B vs. separate streams back to Site A.
This star vs. hub and spoke design has been available in competing solutions over 6 years ago and worked very well (AT&T Connect\Interwise). It would allow a true intersite active\active design. It would require that all frontend pools\servers are aware of all conferencing which raises the frontend server\pool requirements, but I believe the improvements in bandwidth utilization outweighs the increase in frontend pool\server requirements.