Drone shows:
Creative potential and best practices

Citation: Waibel, M., Keays, B., Augugliaro, F.. Drones shows: Creative potential and best practices. Verity Studios, white paper, January 2017, available online via www.veritystudios.com

Abstract

Drone show systems are advanced show automation systems designed for live performances that enable the control of one to potentially thousands of autonomous multirotors or other small flying machines. These novel systems enable the use of drones to extend the traditional palette of light, sound, stage effects, and human performers by choreographing the simultaneous movements of a multitude of flying objects in 3D space. This gives this technology the potential to fundamentally transform the live events experience. Emergent use cases for this technology include animation of flying characters, flying lighting displays, movement of stage props and scenographic elements, and the creation of flying robot actors and synthetic swarms. The key challenges for this new technology are safety and reliability. First solutions for the safe deployment of this technology exist, and avant-garde creators are embracing the possibilities offered by autonomous drones.

Table of contents

  • Introduction
    First examples of drone show systems and their unique requirements
  • Technologies
    Components and requirements for a complete drone show system
  • Key considerations
    Design and operational considerations related to safety, reliability, and ease of use
  • Conclusions
    Summary of creative potential, operational risks, and current solutions

Introduction

Flight evokes powerful emotions, well attuned to the live events experience. The past years have seen the first examples of live drone performances. Examples indoors range from a permanent theatrical installation on Broadway [2] to lighter-than-air vehicles used for a concert tour [3], in addition to multiple one-time events including a theatrical performance of Shakespeare’s “A Midsummer Night’s Dream” [4], TV appearances on America’s Got Talent [5] and Britain’s Got Talent [6], an act by a leading magician [7], and a performance of a synthetic swarm at the TED 2016 conference [8], among others. Outdoors examples include shows by Spaxels [9], CollMot Robotics [10], Puy du Fou [11], Intel [12], and Sky Magic [13].

Drone shows are an emergent use case, distinct from other drone applications like filming and photography. This use case typically calls for multiple drones flying simultaneously, often in formation and often synchronized and deeply integrated with light or projection effects, stage effects, or music. For three or more drones being used simultaneously, the use of human pilots for remotely controlling the drones becomes impractical and often creates heightened safety risks. Instead, drones fly autonomously. Early examples of such autonomous systems use vision-based technologies such as motion capture [1] or use static, structured environments (e.g., floors and/or walls with marker patterns) with well-controlled lighting conditions in combination with onboard cameras for localization [14]. Systems that are currently in operation rely on external, radio-frequency-based positioning systems such as GPS [15] or other radio-frequency-based technologies [2] in combination with onboard receivers to localize in space.

Figure 1: Examples of live drone performances (starting top left): Muse Drone Tour [3], Ars Electronica’s Spaxels [9], Cirque du Soleil’s Paramour on Broadway [2] Raffaello D’Andrea at TED 2016 [8], Intel-Disney ‘Starbright’ Holiday Drone Show [15], ABB’s 125th Anniversary Celebration [17], Saatchi & Saatchi’s Meet Your Creator [16], Elevenplay’s dance with drones on America’s Got Talent [5].

This new use case for autonomous drones also has unique requirements in terms of safety (flight of multiple drones close to crowds of people) and managing operational risks, which include high indirect costs of safety incidents due to brand or reputation damage, in addition to direct human, material, insurance, and legal costs of safety incidents. These risks are further compounded by the exceptional public visibility of live events. In addition to the very high safety requirements, other requirements of this new use case are related to the live event industry’s high demands for reliability and uptime, to the high requirements for ease of use in a high-pressure live event setting, and to guaranteeing low operating and maintenance costs. State-of-the-art drone show systems combine multiple cutting-edge technologies, including intelligent, self-monitoring machines with a high degree of autonomy and built-in redundancies, to meet these requirements.

Artistic value

What emerges from this rich layer of technology is strikingly poetic. Artistic value is derived from the ability to create striking images and sculptures in space, animating their movement and lighting, and tightly synchronizing the intricate patterns that emerge to music, to express emotion and to endow individual or collective, floating characters with personality. The great flexibility of these systems ranges from pixel-level control of motions and lighting to carrying elaborate, actuated costumes for characters that can be freely positioned and moved in 3D space. This allows mimicking and recalling a wide range of emotions that we easily recognize: curiosity, joy, fear, anger, love.

Unlike pixels on screen, the drones are physical objects that can move with intent and purpose. They occupy the same space as human performers and can interact with them through lighting, motion, sound, or physical contact. And they can move into the space of spectators, breaking the fourth wall. Their effect can be surprising and its emotional impact irrepressible. As such, this technology offers a powerful story telling mechanism that connects with the audience to a degree beyond what is possible with conventional stage techniques.

In a show, drones can be used in a variety of ways: They can be dressed up in costumes to act like characters or as three-dimensional scenography; they can dance and interact with human performers; they can become flying lights offering unique lighting opportunities; they can be used to deploy stage effects like confetti, snow, or stage fog; and they can carry payloads including cameras, mirrors, lasers, or spotlights.

Figure 2: Drone costumes (starting top left): Verity Studios, SPARKED [18]; Otto Dieffenbach, Superman Drone [19]; Ctrl.me, Performance Drones [20]; Alan Kwan, Flying Umbrellas [21]; John Cale, Liam Young, Transmissions from the Drone Orchestra [22]; Verity Studios, Cosmic Shell [23]; Oli Metcalfe Design, Muse Drone Tour [3]; Festo, FreeMotionHandling [24].

Large numbers of drones can be used to create natural swarm-like behaviors or geometric shapes. These choreographed movements create novel stage effects that cannot be achieved otherwise. Synchronization of each drone’s lights and movement allows for the creation of intricate patterns, which can be synchronized with stage lighting, effects, and to music. By leveraging human perception of 3D structure from motion, drone swarms can be used for convincing illusions of very large objects floating in space [9]. Synthetic swarms of small, safe drones flying out over the audience have been successfully used to stunning effect [8, 17].

Due to this technology’s novelty, high degree of technical sophistication, and physical limits of drones’ movements, creative ambition must be carefully balanced against safety and reliability considerations. The most compelling results are achieved by engaging with drone show experts early in the creative process.

Technologies

Drone show systems combine a series of cutting- edge technologies to allow a single operator to simultaneously control the coordinated movement of many drones. The drones used as part of such systems are usually equipped with a high level of machine intelligence. Although the use of human pilots is possible in some cases, the requirement of having at least one pilot per drone makes this approach operationally challenging. Pilot errors have been an important contributing factor in many documented drone accidents, as has the necessity of an uninterrupted wireless link for remotely controlling drones from the ground. Safety risks rapidly increase as the number of drones increases, resulting in more complex flight plans and higher risks of mid-air collisions. Autonomous control allows safer operation of multiple drones than remote control by human pilots, especially when operating in a reduced airspace envelope.

In drone show systems, computerized systems therefore replace the pilots. The drones are flying mobile robots and navigate autonomously, piloting themselves, while supervised by a human operator. To navigate autonomously, drones require a reliable method for determining their position in space. In outdoor areas, clear of obstacles, drones can use GPS for their autonomous navigation. However, GPS degrades close to large structures (e.g., tall buildings) and is usually not available indoors. Since degraded GPS may result in unreliable or unsafe conditions for autonomous flight, a replacement (e.g., an Indoor Positioning System (IPS)) must be found for use cases that require flight close to structures that obstruct large parts of the sky and for indoor applications.

In a typical setup, the autonomous drones are supervised by a human operator, who does not control drone motions individually but only issues high-level commands such as “takeoff” or “land”. The operator monitors the motions of multiple drones at a time and reacts to anomalies. Drone motions are typically choreographed and pre-programmed. Passing commands from the operator’s control station to the drones requires one or more wireless communication channels.

Flight planning requires tools for the creation of choreographies for large numbers of drones. Such choreographies must account for the drones’ actual flight dynamics, considering actuator limitations, and for aerodynamic effects, such as air turbulence or lift created by a costume. State-of-the-art tools provide safety and feasibility guarantees for the drones’ trajectories, allow for the rapid and safe generation of collision-free motions for swarms of drones, and may provide real-time trajectory generation algorithms for emergency maneuvers [1].

Typical system architecture for a drone show system

Figure 3: Typical system architecture for a drone show system comprising autonomous drones, a positioning system, and control station(s) including emergency stop circuitry and wireless communications architecture.

Key considerations

The complexity of drone show systems that allow for the autonomous navigation of dozens, hundreds, or thousands of drones in a safety-critical environment and in a live performance setting exceeds the complexity of most advanced industrial automation systems found in state-of-the art factories and warehouses. The only comparable systems that can serve as a template for operating large numbers of autonomous mobile robots with high demands for reliability and safety are those developed by Kiva Systems (now: Amazon Robotics), where thousands of autonomous mobile ground
robots are used for automated storage and retrieval in warehouses [25]. This section summarizes general safety considerations, safety standards, drone show system-specific safety considerations, and specific requirements for reliability and ease of use when deploying a drone show system into the well-oiled mechanism of show productions.

General safety considerations

Like other machinery, drones can cause injury or damage when used without care. Although the authors are not aware of an accident resulting in injury caused by a drone show system, the careless use of drones at live events has the potential for harm, as demonstrated by the recent hand injury suffered by singer Enrique Iglesias [26] while performing live on stage with a remote-controlled camera drone (see Figure 4) and by the crash of a professional, remote-controlled camera-carrying drone that missed skier Marcel Hirscher [27] by fractions of a second during a World Cup race broadcast live to an international TV audience (see Figure 5).

Using a drone show system requires a systematic risk assessment (e.g., according to SORA [28]). For drone shows, the consequences of a safety incident mainly depend on the type of drones and whether they use open rotors, on their size and mass, on the effectiveness of measures to mitigate the risks of a component failure, and on the reliability of the entire drone show system. In addition, risks depend on the type of performance: When used outdoors (e.g., as part of a lighting display in the sky), risks can typically be limited to a small number of people situated below the drone flight paths through sufficiently large horizontal spatial separation, and to the trained drone operators handling the drones before their start, after landing, or during testing and setup. When used indoors (e.g., as part of a stage performance), drones are typically closer to their audience and consequences may therefore also affect members of the audience and performers on or near the stage.

Safety standards

The manned aviation and the live entertainment industry both have well-established safety standards. The emerging drone industry is still in the process of establishing standards. To date, no safety standards exist for drone show systems and autonomous drones, and there are no established safety standards for drones used in the consumer and commercial industries yet. The latest set of regulations issued by the US Federal Aviation Authority (FAA) does not yet list any airworthiness requirements for small drones; responsibility (and liability) lies solely with the drone pilot. European regulations are still being drafted and will be centered on the safety risks posed by drones. Due to the current absence of international standards, some jurisdictions have issued their own drone regulations. These regulations sometimes differ substantially from each other and from current drafts for international regulations. Some drone regulations may also deviate from the recommendations given in the present white paper. Generally, drone regulations, such as those issued by the FAA and their counterparts in other countries, only apply to airspace and do not apply indoors.

Enrique Iglesias' drone accident

Figure 4: A remotely-piloted camera-carrying drone was used as part of a live stage performance at a concert in May 2015 in Mexico. When the lead performer grabbed the drone and pulled it closer to sing into its camera, his hand got caught in the drone’s exposed propellers, resulting in bloody cuts and a fractured hand [26]. The drone was a remotely-piloted consumer drone unsuitable for this type of use at live events.

The current lack of airworthiness requirements in drone regulations has been criticized by the authors of a recent study that investigated 150 drone-related accidents, which showed that technology issues, rather than pilot errors, are the key contributor to these accidents [29], echoing the observations of an earlier study of incidents with military drones [30]. The limits of today’s approaches to safety have also been clearly demonstrated by the long list of safety incidents involving remote-controlled drones at events [26, 27, 31, 32, 33, 34, 35, 36, 37, 38, 39]. Note that this list only includes published safety incidents found during an online search and is therefore likely far from comprehensive. This lack of safety features on today’s consumer and professional drones has also taken many of the amateur and commercial pilots responsible for these safety incidents by surprise, with many commenting that they had not expected that a single failure mid-flight would result in an immediate loss of control and crash to the ground [34, 36].

The current European Aviation Safety Agency (EASA) draft rules consider kinetic impact energy and the potential of damage, such as skin being cut. In a 2016 report [40] to the US Federal Aviation Administration (FAA), the Micro Unmanned Aircraft Systems’ Aviation Rulemaking Committee proposed that small drones be considered safe for flight over people if their total weight (i.e., including accessories/payload such as cameras) was below 250 grams. However, as argued in a recent paper [41], this 250g weight limit may not be well-founded. Moreover, soft tissues such as eyes need special consideration, and a substantially lower total weight limit or strict regulations for encasing rotors may be required to rule out severe eye injury that may be caused by rotating propellers.

Designing drone show systems for safety

Given the lack of safety features on today’s commercially available drones and these drones’ relatively poor safety record [29], a system for performances with multiple, autonomous drones, operating close to crowds must be carefully engineered for safety.

While safety engineering is also required for other complex, safety-critical stage equipment, it is especially critical for drone show systems, because safety cannot be achieved with a “system off” state. For example, while it is usually possible and acceptable to mechanically brake and turn off machinery moving on wheels, tracks, or cables as well as ground robots like those used in warehouses, it is usually not safe to turn off flying machines mid-flight without compromises on their show effect (e.g., very light-weight drones, physical separation from the audience, or both). If a crash is to be avoided, aircraft must continue to function despite any single failure, at least to a level that allows a safe emergency landing. This is why redundancy, i.e., the duplication of critical components and functions of a system, has long been the norm for manned aircraft.

Marcel Hirscher's close call with a camera drone

Figure 5: Skiing champion Marcel Hirscher narrowly escaped being hit by a camera-carrying drone that experienced an unmanaged failure, resulting in an uncontrolled descent and crash during the second run of the World Cup’s slalom event in December 2015 in Italy. The drone was a remotely piloted professional drone. The explanation given by the pilot was signal interference, resulting in a loss of control and subsequent crash [42].

Drone show system designers should therefore strive for a design that does not include any single point of failure. Where a single point of failure is unavoidable, a fail-safe or weakest link design should be used. The failure of any single system component cannot result in an unmanaged loss of control, but must be handled appropriately. For example, a drone should be able to land in a controlled fashion despite a failure of any one of its motors, any part of its electronics, any one of its cables or connectors, and any one of its batteries. Similarly, loss of communication between a drone and its ground station must not create a dangerous situation.

This can be achieved by designing a fully redundant system, i.e., a system that can continue operation, or trigger a safe emergency behavior, if any one of its components fails. While such designs can be more expensive to create than designs without duplication, they provide much higher safety (and help reduce insurance costs). Their decades-long history in the manned aviation industry has shown this approach’s effectiveness and provides a treasure trove of experience and evidence for the emerging drone industry. One way to design such a redundant system is to first design a simple flight-capable system, to then duplicate all its components (e.g. sensors, processors, actuators) to achieve the necessary redundancy, and to carefully design switch-over from a failed system to its backup. Importantly, the design of such a system cannot stop with the design of fully redundant drones, but needs to extend to all other critical components of the drone show system, including the positioning system, communications architecture, e-stop systems, and control stations. For example, localization systems based on external vision sensors (“optical motion capture systems”) that are frequently used in stage environments for non-safety-critical applications and that are successfully used to provide ground truth for robots in an academic research setting are typically not redundant, relying on a single central computer for processing. Such single points of failure at the systems level are particularly critical, because they can cause the simultaneous failure of all drones.

An alternative approach to fully redundant drones is to accept that drones will crash, and to work on decreasing or eliminating the risks of injuries and damage that crashing drones will create. The audience may be protected by containing the drones using protective nets or screens, or by flying drones far away from, and not above, people. Alternatively, safety may be achieved by designing drones that are very lightweight and/or use protective cages to keep the energy of possible impacts to a minimum.

Rotating propellers that power most of today’s drones are a hazard not encountered with most other machinery. While many other machines have rotating parts, drone propellers cannot be fully encased because drones require free airflow into and out of their propellers to fly. Propellers typically rotate at very high speeds and have the potential to cause serious injury to eyes or other human tissue. It is well known from other flight operations that propellers present unique safety hazards since spinning blades are difficult – or impossible – to see and humans lack intuition for rapidly spinning objects [43]. Whenever drones are operated close to people, they should therefore use protected rotors. Care should be taken to mitigate risks from fractured propellers. Flying shards in particular can pose an important eye hazard. Whenever possible, drones should use propellers made of soft material like rubber or soft plastics at the expense of flight efficiency, propeller durability, and operating and maintenance costs.

Unlike other safety-critical stage equipment, drones are susceptible to malicious interference from audience members, as multiple incidents of drones downed by audiences have shown [44, 45, 46]. In addition to physical attacks, other forms of intentional disruption, including hacking or jamming communications, also require careful consideration to reduce the risk of drones crashing into their audience.

In brief, ensuring safety is a complex, multi-facetted, system-level challenge and should be at the center of discussion when developing or using drone show systems.

Designing autonomous drones for reliability

Engineering a system for safety often requires designing for high reliability. Reliability, however, is not only key for safety, but is also fundamental for a successful drone show system that meets the performance and uptime requirements of show productions.

Research labs producing video demonstrations of cutting-edge technology played an important role in the rapid rise in popularity of small drones. For years, videos published online have been showcasing the capabilities of those flying machines, pushing the boundaries of what can be achieved with sophisticated algorithms and control strategies. Academic researchers, however, have a mandate to create proof-of-concept demonstrations rather than deliver reliable, market-ready products. As attested by the comparatively large number of groups producing video demonstrations of their drones versus the very small number of groups who can actually point to a track record of successful live performances or flight hours logged, capturing proof-of-concept demonstrations on video is different from delivering a product. This difference is also clear from the graveyard of failed crowdfunding campaigns [47]: The ability to realize a YouTube video, where brazen marketeers can resort to camera cuts, near-endless tries, and CGI to tell their story, does not easily translate into an autonomous robot product ready for the real world, let alone a drone show system comprising dozens of autonomous entities capable of safely and reliably performing in front of a live audience.

Guaranteeing safety and reliability of drone show systems imposes design constraints that substantially differ from those of consumer drones. Current off-the-shelf drones are unsuitable for drone show systems because these platforms are not designed with a strong focus on reliability, but meant for the occasional weekend flight or for amateurs that choose drone building and repair as a hobby. For example, in a daily show with 10 drones, a 1% failure rate per drone flight would result in one or more drone failures most weeks. As another example, leading consumer drone motor manufacturers tout mean times between failure (MTBFs) on the order of hundreds of hours, resulting in drone MTBFs in the tens of hours (e.g., a quadcopter with individual motor MTBFs of 160 hours [48] will have a MTBF of 40 hours before factoring in other possible component failures [49]). Even high-end motors used in drones starting at USD 10,000 only improve failure rates by one order of magnitude [50, 51, 52]. These numbers are many orders of magnitude below most other consumer goods [53], which are produced by decade-old industries that can benefit from billions spent on engineering efforts and that face less stringent performance and weight constraints than consumer drones.

Similar challenges exist for other key parts of drone show systems. For example, localization technologies like optical motion capture and other vision-based technologies that are being used in a show context for non-safety-critical applications suffer from similar shortcomings, having been designed for applications with different operational constraints. In many cases, these systems are susceptible to light interference (direct stage lighting, light reflections, critical disruption or permanent damage by laser pointers) and cannot tolerate even slight relative movements between cameras, adversely affecting their reliability in a live show setting. Similar safety and reliability concerns are also encountered when using system architectures that require an uninterrupted wireless communication channel between control stations and drones. An architecture commonly used with motion capture systems in a research context, for example, requires uninterrupted, high-speed, low-latency wireless communication for sending the position and orientation data computed offboard to the drones. While this is convenient for research and development, architectures that rely on single critical links are not an acceptable choice for safety-critical applications. More generally, drone show systems should avoid reliance on communication protocols that are highly susceptible to wireless interference, such as Wi-Fi.

Next to reliability, live events also impose other important design constraints that are less pronounced or absent in many other industries. Drone show systems rely on high-precision devices and their running costs, including operation (e.g., charging batteries, calibration, testing, and show operations), maintenance, and repair, can be significant for permanent show installations. Such costs require careful consideration on par with that used for other, comparably complex show equipment.

Overall, for drone show systems reliability is not optional. As part of a live event, drones, like performers, must deliver every single performance, and their behavior must be repeatable and predictable. In addition to its safety benefits, high reliability can also go hand in hand with major reductions in running cost.

Ease of use

Over the past decades the live events industry has widely adopted complex automation systems. Today, automation systems and show control software that allow managing a large variety of automation equipment and media content within a show are integral parts of every large production. Some of the world’s most complex automation systems have been deployed in the entertainment industry and are being used at large theme parks or to run complex touring shows. Drone show systems need to integrate into this ecosystem and fit smoothly into the existing work-flow of automation operators, allowing for easy cuing as well as start and stop behavior. Ease of use also significantly contributes to safety by helping avoid operator mistakes and making the system’s reaction to changes predictable.

Furthermore, to maximize the creative potential and minimize the overhead associated with the use of drones in a show, setup and operation must be straight-forward. It should not require a large crew. Drone show systems should not interfere with, nor be affected by, other technology commonly used on stage or by the show’s audience, including Wi-Fi, wireless microphones, stage effects, or stage lighting. Drone show systems require flexibility to react quickly to unexpected conditions on site. Finally, to fully reach their creative potential, these systems must allow for rapid iterations during show creation and rehearsals, and ease of use can especially contribute to safety during this critical phase. Experienced providers can advise on integration and ease of use, in addition to installation, operation, and maintenance.

Conclusions

Drone shows are emerging as a potent medium of expression in live performance. This novel use case for drones has requirements that are clearly distinct from those of consumer drones and requires a new breed of drone systems that meet the stringent demands of the live entertainment industry in terms of safety and reliability.

Today’s commercially available drones are not designed for use at live events without additional safeguards. Nearly all systems available today contain multiple single points of failure and rely on consumer-grade components that are not designed for high reliability. Such systems have already caused a string of accidents resulting in injuries at events, albeit outside the drone show context [26, 27, 31-39].

The design of drone shows must account for the compounded safety risks encountered in this use case that include:

(1) The simultaneous use of multiple drones, which increases the number of drone failures per performance proportionally to the number of drones used. It also creates the possibility for mid-air drone collisions, which can rapidly escalate a single failure and result in unexpected behavior
of multiple drones (i.e., cascading failure events). In addition, performers and audience members cannot reliably track more than 3-5 drones simultaneously, making it difficult for them to avoid, or to protect themselves against, a falling drone.

(2) Drone shows require complex infrastructure in addition to that used for human-piloted, remotely controlled drones. A safety and risk assessment is essential and needs to include the entire automation system, including control station(s), emergency stops, positioning system(s), and communication link(s), in addition to the drones themselves.

(3) Drone shows typically involve flight close to crowds of hundreds or thousands of people, greatly increasing the likelihood of an out-of-control drone injuring a person. Safety and reliability are not optional in this context. Moreover, incidents involving drones have high direct (e.g., liability) and indirect costs (e.g., reputation, brand damage).

First drone show systems exist and some of these systems have established substantial track records [1, 2, 9, 15, 54]. The key success factors in these solutions include maintaining sufficient safety distance between the drone(s) and the audience and performers; physical separation (i.e., safety nets between the drone(s) and the audience and performers); the use of lighter-than-air drones; the use of ultralight-weight drones (e.g., <100g); and the use of fully redundant systems (i.e., systems without a single point of failure). Given the complexity of this emerging use case and novelty of the technologies employed, show creators should proceed with cautious enthusiasm as they explore the nearly untapped potential of autonomous flying objects.

Appendix: Best practices

System-related questions

What is the most critical component of the system? What happens when it fails?
No single failure in a drone show system should jeopardize safety. Drone show systems should tolerate single failures or allow for graceful degradation of the system’s performance (e.g., some drones land earlier than planned). Failures in mains power, communications (e.g., due to interference), or a software or hardware glitch on the control station should not result in a crash.

How is the system protected against human error? For example, what happens if a drone is placed at a wrong takeoff location? What if a drone is damaged?
Drone show systems should have mitigation strategies to guard against common human errors. For example, the system should automatically check its health and verify each drone’s location before takeoff.

What is the track record of the system? What is the total number of flight hours logged? How many accidents occurred in this period? How many incidents?
While statistics should always be taken with a grain of salt, the number of autonomous takeoffs, flights, and landings as well as total operating hours give an indication of a system’s safety and reliability. A vendor’s transparency towards their system’s safety record can be telling. Use data is also important for safety risk cases and for reducing insurance costs.

What measures can be taken by the operator of the system in case of an emergency?
Unless physically separated (e.g., nets, glass walls) from operators, performers, and audience, even highly sophisticated autonomous machines should be supervised by a skilled operator to detect emergencies that can not be detected automatically. The types of measures available to the operator will affect the safety of the system and minimize the impact of technical issues and unanticipated onstage events on the show.

How is the system secured against intentional disturbance? Can it be easily disrupted or hacked?
Safety-critical systems should reject external disturbances and fail safely in case of intentional disruption. For example, positioning systems or drone navigation based on vision technologies may be susceptible to interference by a laser pointer. Radio-frequency-based systems may be susceptible to pocket-sized signal jammers. Unsecured communications may be easily hacked.

What measures are used to guarantee continued safety of the drone show system’s components?
Drones are high-performance machines that require regular inspection and maintenance for safe continued operation. Localization sensors are thoroughly calibrated, high-precision devices. A detailed maintenance and inspection schedule should be in place.

Drone-related questions

What happens when a component fails? Will the drone crash?
A drone should not crash because of a single drone component failure. Instead, it should react appropriately (e.g., perform a controlled landing). Critical components whose failure should not cause a crash include: Sensors (e.g., IMU, GPS, barometers, cameras used for navigation), effectors (including motors, propellers, and motor controllers driving them), connectors, batteries, processors, software, communication modules (e.g., Wi-Fi). An alternative approach to fully redundant drones is to accept that drones will crash, and to work on decreasing or eliminating the risks of injuries and damage that crashing drones will create (e.g., by installing protective nets or by using very light-weight drones).

How much does each drone weigh? How fast does each drone fly? How high does each drone fly?
The severity of a drone collision must be taken into account when assessing the consequence of a crash. For blunt impacts, severity depends on the drone’s kinetic energy on impact, which is determined by its weight and the speed at the time of impact (i.e., flight speed, falling speed, or both).

Are the propellers on the drone safe?
Propellers pose two main threats: They can slice human tissue, and they can burst into shards on impact, which can pose an important safety hazard to human eyes. Exposed propellers should be avoided (e.g., use a prop guard, cage). All propellers should have been assessed for their shard risk.

What kind of self-checks does the drone perform before takeoff? While flying?
It is always better to prevent a failure than to react to it. Drones should perform a comprehensive, automatic self-check before takeoff (e.g., ensure consistent sensor readings and correct motor function before takeoff). Drones should continuously monitor system health during flight (e.g., battery running low). When a problem is detected during flight (e.g., a motor failure), a contingency plan should allow for a safe landing. Importantly, drones should automatically detect catastrophic events (e.g., a propeller hitting an obstacle, the drone hitting the ground) and automatically turn off and immediately stop their propellers.

What are the safety margins on drone motion? What do they account for?
Varying conditions (e.g., temperature, air flow from ventilation systems, wind) may affect the flight path. Drones that have encountered a failure or degradation may have different flight properties that must be accounted for.

Contributors

This white paper was authored by Verity Studios, a spin-off company of the Flying Machine Arena [1] at the Swiss Federal Institute of Technology (ETH Zurich). We have designed drone systems and used them in shows and live events for the past 8 years and are now commercializing turnkey systems for live events.

We believe that drones will fundamentally transform the live events experience. By sharing the physical space of performers and audiences, drones become a three-dimensional tool that is more intimate than today’s on-screen displays. When equipped with costumes and lights and directed to move with purpose, drones display emotion and intent, becoming actors in their own right. We believe that leading creators will reimagine old characters and bring memorable new flying characters to life, and we are excited about filling the large empty spaces found above today’s audiences with small, safe drones to stunning effect.

Our intent with this document is to contribute to the emerging ecosystem of drone shows and to help accelerate this industry’s safe development. As an appendix to this paper, we include a list of questions to help new developers and users articulate key issues related to drone shows.

We welcome all feedback at droneshows@veritystudios.com.

Authors

Dr. Markus Waibel, Verity Studios

Bill Keays, Verity Studios

Dr. Federico Augugliaro, Verity Studios

Contributors and reviewers (in alphabetical order)

Adam Davis, Chief Creative Officer at Tait Towers

Alec T. Quaite, Aerospace and aviation advisor

David Benken, Technical Director for multiple Broadway and touring productions

Jean-Francois Bouchard, Chief Creative Officer at Cirque du Soleil

Michael Shepp, Automation Carpenter and drone operator for Cirque on Broadway’s Paramour

Mike Davis, Senior Vice President/Executive Producer at Universal Studios, Japan

Markus Farner, Chairman of JARUS WG-31, Chairman of the FOCA RPAS working group2, Avionics Certification Engineer at the FOCA

Prof. Mark W. Mueller, Professor at the University of California, Berkeley

Prof. Raffaello D’Andrea, Professor at ETH Zurich, founder of Verity Studios, co-founder of Kiva Systems (now Amazon Robotics)

References and Footnotes

Footnotes

1 Joint Authorities for Rulemaking on Unmanned Systems (JARUS) Airworthiness Working Group

2 Swiss Federal Office for Civil Aviation (FOCA) Remotely Piloted Aircraft System (RPAS) Working Group

References

[1] Institute for Dynamic Systems and Control, ETH Zurich. (2016) Flying Machine Arena. [Online]. Available: http://­flyingmachinearena.org

[2] Cirque du Soleil. (2016) PARAMOUR; Flying machine design and choreography by Verity Studios. [Online]. Available: http://­veritystudios.com/­about-us/­

[3] Oli Metcalfe Design. (2016) Muse drones tour. [Online]. Available: https://­www.youtube.com/­watch?v=LnfSL2ca3ag

[4] C. Squatriglia. (2009) Robots Perform Shakespeare. [Online]. Available: https://­www.wired.com/­2009/­11/­robots-perform-shakespeare/­

[5] Elevenplay. (2016) America’s Got Talent. [Online]. Available: https://­www.youtube.com/­watch?v=1CjX1r-gh8U

[6] D. Walker/Mirror. (2016) Britain’s Got Talent act Flying Bebop BANNED from recording semi-final live for technical reasons. [Online]. Available: http://­www.mirror.co.uk/­tv/­tv-news/­britains-talent-act-flying-bebop-8042906

[7] M. Tempest/MagicLab. (2016) 24 Drone Flight. [Online]. Available: https://­www.youtube.com/­watch?v=B4xtsH6pzoM

[8] R. D’Andrea/TED. (2016) Meet the dazzling flying machines of the future. [Online]. Available: https://­www.ted.com/­talks/­raffaello_d_andrea_meet_the_dazzling_flying_machines_of_the_future?language=en

[9] Ars Electronica. (2016) Spaxels. [Online]. Available: http://­www.spaxels.at/­

[10] CollMot Robotics/Dancing with Drones. (2015) Dancing with Drones show at Sziget. [Online]. Available: http://­dancingwithdrones.com/­

[11] Puy du Fou. (2014) Neocopter. [Online]. Available: http://­www.puydufou.com/­

[12] Intel. (2016) Intel’s 500 Drone Light Show. [Online]. Available: https://­iq.intel.com/­500-drones-light-show-sets-record/­

[13] Sky Magic. (2016) Sky Magic live at Mt. Fuji. [Online]. Available: https://­magic.microad.co.jp/­skymagic/­movies/­movie2-fuji.php

[14] BeTomorrow. (2016) Flying Bebop. [Online]. Available: http://­www.flyingbebop.com/­

[15] Intel Newsroom. (2016) Intel, Disney light up the sky over Walt Disney World resort with new Starbright Holidays drone show. [Online]. Available: https://­newsroom.intel.com/­news-releases/­intel-disney-starbright-holidays-drone-show/­

[16] Saatchi & Saatchi. (2012) Meet Your Creator. [Online]. Available: http://­saatchi.com/­en-us/­news/­saatchi__saatchi_new_directors_showcase_2012

[17] Verity Studios. (2016) ABB 125 Anniversary VIP Gala. [Online]. Available: http://­www.veritystudios.com/­

[18] Verity Studios, ETH Zurich and Cirque du Soleil. (2014) SPARKED. [Online]. Available: https://­www.youtube.com/­watch?v=6C8OJsHfmpI&t=4s

[19] O. Dieffenbach/Reuters. (2016) Superman drone. [Online]. Available: http://­msnbcmedia.msn.com/­j/­MSNBC/­Components/­Photo/­_new/­pb-130628-superman-da-01.photoblog900.jpg

[20] Ctrl.me. (2016) Performance drones (hula hoop light show). [Online]. Available: http://­www.wetalkuav.com/­wp-content/­uploads/­2016/­10/­Screen-Shot-2016-10-20-at-12.05.35-AM.png

[21] A. Kwan. (2016) Flying umbrellas. [Online]. Available: http://­www.popsci.com/­sites/­popsci.com/­files/­styles/­xl_1x_/­public/­Digital_Synesthesia_Kwan_Alan.jpg?itok=wwbag1R5

[22] J. Cale and L. Young. (2014) Transmissions from the drone orchestra. [Online]. Available: https://­www.droneuniversities.com/­wp-content/­uploads/­2015/­07/­a-drone-orchestra-takes-flight-a.jpg

[23] Verity Studios. (2016) Costume development lab. [Online]. Available: http://­www.veritystudios.com/­

[24] Festo. (2016) Autonomous flying gripping sphere. [Online]. Available: https://­www.festo.com/­group/­en/­cms/­11957.htm

[25] B. Einstein/Robohub. (2012) Meet the drone that already delivers your packages, Kiva robot teardown. [Online]. Available: http://­robohub.org/­meet-the-drone-that-already-delivers-your-packages-kiva-robot-teardown/­

[26] Agence France-Presse in Los Angeles. (2015, June) Drone injury grounds Enrique Iglesias for longer than expected. [Online]. Available: https://­www.theguardian.com/­music/­2015/­jun/­03/­drone-injury-grounds-enrique-iglesias-for-longer-than-expected

[27] ——. (2015, December) Drone crashes onto piste, misses champion skier by inches. [Online]. Available: http://­edition.cnn.com/­2015/­12/­23/­sport/­marcel-hirscher-drone-crash/­

[28] JARUS, “JARUS guidelines on Specific Operations Risk Assessment (SORA),” 2016.

[29] G. Wild, J. Murray, and G. Baxter, “Exploring Civil Drone Accidents and Incidents to Help Prevent Potential Air Disasters,” vol. 3, no. 3, p. 22, 2016. [Online]. Available: http://­www.mdpi.com/­2226-4310/­3/­3/­22

[30] K. Williams/Office of Aerospace Medicine Washington, DC. “A Summary of Unmanned Aircraft Accident/Incident Data: Human Factors Implications,” Civil Aerospace Medical Institute – Federal Aviation Administration, December 2004.

[31] Associated Press/Daily Mail. (2015, December) Two people are injured after a drone taking aerial video of a race fell into the crowd of 10,000. [Online]. Available: http://­www.dailymail.co.uk/­news/­article-3341510/­Drone-falls-crowd-injures-runners-road-race.html

[32] BBC News/ABC News. (2014, April) Australian triathlete injured after drone crash. [Online]. Available: http://­www.bbc.com/­news/­technology-26921504

[33] B. Vincent/Engadget. (2016, June) This DJI Phantom 3 drone hit a Quebec woman in the head. [Online]. Available: https://­www.engadget.com/­2016/­06/­24/­phantom-drone-hit-woman/­

[34] G. Cruz/Eagle Tribune. (2015, December) Feaster Five drone crash under investigation; device hit tree, runners during Thanksgiving road race. [Online]. Available: http://­www.eagletribune.com/­news/­merrimack_valley/­feaster-five-drone-crash-under-investigation/­article_b05462d5-e915-50d4-b322-c04dfafe91cd.html

[35] H. Rush KRQE News. (2015, June) Drone hits person at Albuquerque festival. [Online]. Available: http://­krqe.com/­2015/­06/­14/­drone-hits-person-at-albuquerque-festival/­

[36] M. Weil/Washington Post. (2013, August) Drone crashes into Virginia bull run crowd. [Online]. Available: https://­www.washingtonpost.com/­local/­drone-crashes-into-virginia-bull-run-crowd/­2013/­08/­26/­424e0b9e-0e00-11e3-85b6-d27422650fd5_story.html

[37] V. Ogle/Brooklyn Daily. (2014, December) Drone cuts off tip of photographer’s nose. [Online]. Available: http://­www.brooklyndaily.com/­stories/­2014/­50/­bn-drone-disaster-at-tgifridays-2014-12-12-bk_2014_50.html

[38] E. Reagan/Photography Bay. (2015) Fox news guest crashes drone while talking about drone safety. [Online]. Available: http://­www.photographybay.com/­2015/­02/­07/­fox-news-guest-crashes-drone-while-talking-about-drone-safety/­

[39] K. Blessing/Eagle Tribune. (2016) Wedding drone crash leads to guests’ lawsuit. [Online]. Available: http://­www.eagletribune.com/­news/­wedding-drone-crash-leads-to-guests-lawsuit/­article_9bdf14d4-c3bd-11e6-87dc-0752f3c938b6.html

[40] G. Mortimer/sUAS News. (2016) FAA Micro Unmanned Aircraft Systems Aviation Rulemaking Committee (ARC) Recommendations Final Report. [Online]. Available: https://­www.suasnews.com/­2016/­04/­faa-micro-unmanned-aircraft-systems-aviation-rulemaking-committee-arc-recommendations-final-report/­

[41] W. Stockwell and B. Schulman. (2016 (unpublished)) Defining a lowest-risk UAS category.

[42] Infront Sports and Media. (2015) Second statement regarding FIS Alpine Ski World Cup, slalom race, on 22 December 2015. [Online]. Available: http://­www.infrontsports.com/­news/­2015/­12/­updated-statement-%E2%80%93-regarding-fis-alpine-ski-world-cup,-slalom-race,-on-22-december-2015/­

[43] C. Rash/Aerosafety World. (2006) Walking into trouble. [Online]. Available: https://­flightsafety.org/­­asw/aug06/asw_aug06_p28-34.pdf

[44] Runnerazzi. (2014) Drone knocked down at staples center during LA kings stanley cup celebration. [Online]. Available: https://­www.youtube.com/­watch?v=KR39DErszRE

[45] J. Robinson/MailOnline. (2016) Bullseye! russian man frustrated by ’annoying’ drone buzzing actors at medieval battle reconstruction takes the quadcopter down with a perfect throw of his spear. [Online]. Available: http://­www.dailymail.co.uk/­news/­article-3584450/­Bullseye-Russian-man-frustrated-annoying-drone-buzzing-actors-medieval-battle-reconstruction-takes-quadcopter-perfect-throw-spear.html

[46] D. Peters/MailOnline. (2015) What a shot! student lands a perfect kick and takes out a drone flying over a soccer pitch – but ruins the promotional video his school was trying to film. [Online]. Available: http://­www.dailymail.co.uk/­news/­article-3296647/­New-Zealand-student-ruins-promotional-video-kicking-soccer-ball-drone-owned-KOR-Creative.html

[47] K. Perlongo/Robohub. (2016) Why do robotics crowdfunding projects fail? [Online]. Available: http://­robohub.org/­robohub-roundtable-why-do-robotics-crowdfunding-projects-fail/­

[48] T-motor. (2013) U-Power. [Online]. Available: http://­www.rctigermotor.com/­html/­2013/­news_1008/­8.html

[49] J. Jones, Integrated Logistics Handbook, 2006.

[50] PowerVision Technology Ltd. (2015) PowerVison UAV. [Online]. Available: https://­en.wikipedia.org/­wiki/­PowerVision_UAV

[51] Microdrones Gmbh. (2017) Microdrones. [Online]. Available: https://­www.microdrones.com/­en/­company/­dealers-and-distributors/­why-microdrones/­

[52] Moog Inc. (2016) Rotary servo actuators. [Online]. Available: http://­www.moog.com/­literature/­MCG/­APG4n5pdf.pdf

[53] Alion System Reliability Center. (2001) Typical equipment MTBF values. [Online]. Available: https://­src.alionscience.com/­pdf/­TypicalEquipmentMTBFValues.pdf

[54] AirStage. (2016) Air entertainment. [Online]. Available: http://­airstage.de/­?lang=en

Share This