Any Given Sunday
Part rock concert, part plenary session, part live broadcast, part multicast stream… little wonder Hillsong does things differently.
31 July 2016
Story: Christopher Holder
Every Sunday an enormous AV production and broadcast machine kicks into action. It’s as sophisticated and as disciplined as any in the region. It’s a church. But not just any church, it’s Hillsong.
Everyone’s heard of Hillsong. If you’re aligned to a church you’ll know the Hillsong music, and be aware of its huge conferences. If you’re in the AV business you’ll know they’re a big consumer of product. (If you’re neither in the business nor in a church you’ll have seen Hillsong in the mainstream media.) But unless you’re an integral part of the Hillsong team you don’t know the half of it.
Hillsong is going gangbusters… internationally gangbusters. There are 15 permanent campuses in Australia and a growing number overseas, from LA to Kiev.
It all emanates from Hillsong’s home base in Sydney’s Hills district where the 3500-capacity auditorium heaves three times a Sunday to a slickly packaged combination of brilliantly presented contemporary worship music and inspirational messages from the pulpit.
Each service requires the work of dozens of staff and volunteers: audio, lighting, comms, broadcast, video, camera operators, graphics, CCU, tech direction, production… it’s an enormous undertaking from an array of individuals each committed to getting the best result and doing even better next time.
After receiving Adamson’s ‘Worldwide Sound Reinforcement Partnership’ press release a little while back, AV Asia Pacific decided it was high time to pay Hillsong another visit, our first since the church installed an Adamson Y10 rig in 2011.
So first, let’s look at the big Adamson news. We sat down with Steve Le Roux, Hillsong’s Technical Facilities Manager for some background.
ADAMSON & EVE
Steve Le Roux: “Upgrading to the Adamson Y10 changed everything [start of 2011]. Up until that point, the system we had wasn’t delivering and it was hurting the team. It was a difficult PA to mix on because it performed differently at one sound pressure level to another level. You couldn’t train and develop people because it was almost like an old gearbox without synchronisation, where you had to build up the revs but you’re grinding away regardless.
“The problem was a quirk of the product, the amplification, and the fact the array was too short for the space, resulting in holes in the coverage — as you walked from stage to the mix position you went through maybe six patchy zones. It really was a difficult system. So to use the gearbox analogy again, moving to the Adamson Y10 was like going from a manual gearbox without synchromesh to a smooth automatic.
“That upgrade also built up a lot of trust between leadership and the tech team. The system met all expectations, we got it in on budget and the church just lifted — the worship experience was so much better and the message was heard everywhere.”
HANGING OUT WITH ADAMSON
The Hills campus auditorium is cavernous and notoriously tricky in the low end. The 45–52Hz area can lose definition and get away from you pretty quickly.
The Adamson E119s single-19 subs are flown in a position that ensures perfect coupling with both the main hang and the side hang. The theory goes that if you arrayed the E119s alongside the main hangs you would have perfect coupling to the front but once you move to side the path lengths change, and you won’t get proper coupling with the side hangs. By positioning the subs in a way that the distance between the subs and the two hangs is the same overcomes this issue.
Rounding out the system (and part of the original Y10 install) are six Adamson T21 subs (two hangs of three) in end fire mode. Not you’re everyday end-fire configuration, they use a proprietary Adamson preset that provides for more consistent real world damping across the audio bandwidth as opposed to great damping in one frequency range and not so much in others.
MEET THE ADAMSON E SERIES
The Adamson E Series is made up of two full-range cabs: the E12 and the E15. The 12s and the 15s share the same co-linear mid/high section (based on a seven-inch kevlar cone and a four-inch compression driver) and the same crossover frequency. The E12 has a wider and higher (110° x 8°) dispersion compared to the longer-throw (90° x 6°) E15. The E15 is suited to large-scale applications, while Adamson recommends the E12 for use up to a throw of around 80m. With a maximum throw distance of some 50m, the Hillsong system uses a 12-box main hang of E12s.
Adamson manufactures most of its transducer designs, and the 19-inch driver of the E119 and E219 subs represents the company’s push to improve performance and reduce the size of the driver and weight… and also not worry too much about whether it’s an 18 or a 21-inch driver.
The E219 is designed with a high impedance so you can run two in parallel, something you couldn’t do with the legacy T21 — mainly because there wasn’t an amp at the time of its release (2005) that could deliver that kind of performance in a 4Ω load.
Nowadays, a single Lab.Gruppen PLM20K44 can run three E219 cabs, resulting in cost savings. Adamson now exclusively recommends Lab.Gruppen amps.
SHOOTOUT HEAVEN
Selecting the Y10 came as a result of an exhaustive series of system tests.
The Hillsong tech team invited the big players to temporarily fly a rig in the Hills campus, for a week or so at a time, whereupon all the key operators had a chance to mix a service. Scorecards were handed out among the tech team, the musicians and the leadership.
According to Hillsong’s Steve Le Roux and Hillsong’s technical infrastructure leader, Ricki Cook, Adamson was a clear winner.
This time around, a more modest shootout was staged at the church’s Waterloo campus for a smaller line source solution. The Adamson S10 won the day.
Adamson’s Australian distributor CMI, in a savvy commercial move, kicked off a conversation with Hillsong that led to a deal for Adamson to supply the church’s global PA requirements. The idea was attractive to Hillsong as it could standardise its PA technology, which makes the rollout of new campuses just that little bit easier — not only to design and install but in the training of operators.
The deal included an E12 upgrade of the main campus system. When the Y10 rig was installed, the Y Series was soon to be superceded by the E series which leverages Adamson’s most recent driver and waveguide designs.
The new system is based on two main hangs of 12 x E12 with side hangs of 10 x E12. Spaced dead centre between the main and side hangs are three E119 subs in a cardioid configuration (the middle box is facing backwards). There are a further six T21 subs flown in the centre (in an end-fired cardioid configuration) for further LF reinforcement [more on how the system is configured next page].
BROADCAST DEMANDS
Hillsong’s broadcast capabilities are formidable. (In fact, it has recently launched a 24-hour Hillsong channel on the TBN Network, broadcasting globally, but that’s another story.) The broadcast capability started with the need to record services for online streaming. Once multiple cameras became a reality, the broadcast suite became increasingly sophisticated with vision switchers, multiple video monitors with picture-in-picture capabilities, a Euphonix digital mixing console for the audio mix, overseen by a producer calling the shots.
The team would hone its skills on producing special events packages and the production values would increase year on year.
With new campuses due to go online the Hillsong leadership team threw Ricki Cook and the tech team a curve ball: ‘How can we multicast the Hills campus sermon to the satellite Hillsong churches in Sydney, and even Melbourne, Brisbane and elsewhere?’
MULTICAST OF THOUSANDS
Here’s an example of how a Sunday morning might play out between the Hills HQ and the two Sydney City campuses:
The Hills campus’ first service starts at 9.15am, the Alexandria campus starts at 9.30 and Waterloo at 9.45am. To share the sermon with the two city campuses (where the AV package would be beamed onto the other sites’ screens and PA), Hills would record the service then play it back at the appropriate point.
While it’s recording, teams in the Alexandria and Waterloo campuses will be watching the feed from HQ. Those satellite teams, in their own respective broadcast suites, can record, playback and cue the recording. The satellite team member will find the jump-in spot and use that as the cue point.
Having a system that simultaneously takes care of record and playback was a technical hurdle to jump.
But to ensure that having Pastor Brian Houston dropping into your church didn’t feel like someone pressing Play on a VCR, music from the Hills HQ is also piped in. Here’s how that works:
If Pastor Brian unexpectedly throws to a song, then the local music team can play along with the Hills worship team. Each audio operator will bring up the Band stereo stem, Leader and Vocalist stems. The local monitor engineer will already have brought up the click and cue stem (shout mic audio from the music director). The remote campuses can then jump in, with the band taking their cues from the Hills team and the FOH operators mixing as much or as little of the local team as they like through the house PA.
It’s a clever approach, especially when you add a bunch of other Hillsong campuses and the ability to pull off the same trick in reverse (ie. the Waterloo or Alexandria campus sending a feed with eight channels of audio to the Hills HQ).
ADAMSON S10: THE CAB THAT COULD
The S10 is a more recent addition to the Adamson lineup and a response to the market for: something smaller (it’s a two-way), more flexible and more cost effective, that could happily be driven from a minimal number of amp channels (as many as eight cabinets per amp), but still be a true line source.
Like the E12, the S10 provides a 110° horizontal coverage and has the same sonic signature as the E12 above 2kHz (with its four-inch titanium compression driver).
The S10 doesn’t have the nifty colinear drive module of the E12 but has the same waveguide (to ensure phase alignment of the mids and highs).
Unconventionally, the 10-inch drivers splay outwards to provide the symmetrical horizontal dispersion.
Prior to the Hills campus shootout, Hillsong pulled the trigger on a number of S10 purchases for its Waterloo and Brisbane campus.
THROWING OUT STREAMERS
As you might expect, Ricki Cook has given the streaming setup much consideration. As the level of complexity rose he sought to simplify matters to ensure his Sunday tech teams (largely composed of volunteers) could get a result and deal with the unexpected without meltdowns. “Sometimes the streaming schedule would change unexpectedly and the volunteer would have to phone a tech to change the source of the stream.” Instead Ricki and his team use a program called VSM, which allows him to design a simple UI where the operator can choose the source (either one of the Hills venues or a link from another campus with broadcast capabilities). The interface conveniently includes a small video live feed window and audio meters for extra confidence.
VSM replaces banks of hardware video source/destination switches, with each VSM preset taking care of some really quite involved re-routing routines in the background: “That one apparent change to a video router sets off a chain of commands that will automatically change the audio router, Lawo VPros might be re-embedding everything onto the SDI feed, and then back to the streaming servers and up onto YouTube. That’s a massive undertaking in the background all from one button press on a touchscreen.
NETWORKING: JUST PLUG IN
Ricki Cook is also Hillsong’s network guru. With AV accounting for the majority of network traffic, he argues that he needs to be. We began discussing networking while up in the gantry 15m above the platform looking at the lighting — which runs on the Ethernet-based Pathport protocol. Ricki Cook: “The MA Lighting GrandMA2 runs Artnet to the main Pathnet node. It then runs on our existing Ethernet infrastructure. The Ethernet plugs in then breaks it out as two universes of DMX. When you hang a fixture on one of those nodes, you select which universe you want to output. It makes it really easy.”
Ricki Cook has plans to consolidate his networking even further. He’s using the current build of the Epicentre facility as a networking proving ground:
“The next building goes one step further. At the moment you have to plug a fixture into a dedicated lighting network — the Lighting VLAN. In the new Epicentre you’ll be able to plug into any Ethernet port you like. The network will recognise the device; authenticate it; and put it in the right VLAN for it to start working.”
This approach to networking will be a godsend. It means rooms can be especially versatile. For example, the Epicentre will have spaces that serve as lecture rooms during the week and as a small band rehearsal space of an evening or the weekend. Rick Cook again: “The musicians can wheel in a compact rack with a couple of Yamaha Rio interfaces on Dante. They can pull out a couple of Ethernet cables that will plug into the nearest Ethernet port and the network will auto-provision that. It removes the complexity.”
It almost sounds too good to be true, but Ricki has been playing with SDN (software defined networking) for some time: “I’ve been working with a system from Avaya. Once you plug a device into an Ethernet port it sends an identifying Ethernet frame out; the network switch picks it up, detects the MAC address, looks it up in a database — ‘you’re a Yamaha Rio and you belong to this particular VLAN’ — and if everything lines up, it automatically drops it in. It means that as we buy Ethernet devices we take the MAC address, turn it into a QR code, stick it on the device and when someone wants to authenticate that device in a different venue or room we can scan that QR code on a smartphone, type in an admin password, tell it which VLAN it needs to be on, authenticate it, give it a minute, and bang, it’s up.
“This approach is great for our tech team. You don’t have to phone the IT department: ‘I’ve just plugged into Port 43, I think, but it’s not coming up… can you switch that to VLAN 601 for me?’ And the poor help desk guy is thinking ‘I don’t know if I should do that’. Instead he’ll create a ticket, escalates it up until it hits a service engineer. Which all takes time. This way, as soon as you unplug a device, the port shuts down and resets back to a zero state, which mean you can plug in a TV or whatever the next day, even an AMX controller — the network will recognise it and put it in the AMX VLAN.”
SAME HY MN BOOK
It’s fascinating to see an institutional tech rollout of this magnitude and sophistication. Hillsong really is writing its own AV production and broadcast playbook, as its needs are unique. Yes, there are bigger AV productions; yes, there are more complicated live and recorded broadcast gigs; yes, Hillsong isn’t the first to adopt software defined networking… but it’s arguably Robinson Crusoe when it comes to combining all these disciplines week-in, week-out across multiple sites.
RESPONSES