This paper was presented by Tzu-Pei Chen at the 1992 AUSROC conference, Adelaide, Australia, 9-11 December 1992. AUSROC II : A Post Mortem ~~~~~~~~~~~~~~~~~~~~~~~~~ Tzu-Pei Chen Abstract - The AUSROC II Amateur Rocket malfunctioned at launch. The LOX valve failed to open fully, preventing the rocket from lifting off. Pneumatic and electrical umbilicals burnt through preventing an abort sequence. An internal fire started in the lower valve fairing and spread throughout the rocket, eventually destroyed the payload. A design fault in the pressurisation mechanism allowed oxygen to enter the kerosene tank resulting in an explosion which destroyed the vehicle. No definite reason for the LOX valve failure has been found, but a seal failure in the LOX valve vane actuator seems the most likely cause. Simple changes to both the rocket and launcher systems could have prevented further damage to the vehicle after the LOX valve failure. A second vehicle designated AUSROC II-2 will be built incorporating these changes. This paper describes what is known about the launch event. It proposes possible reasons for the failures which were encountered, and suggests solutions where possible. I. INTRODUCTION ~~~~~~~~~~~~~~~~ "If one part fails the whole thing can fail. It's not like a car, if you get a flat tire, you stop and put another one on ... if you blow a valve, you'll probably blow up your tanks and everything along with it." Mark Blair, March '92 On October 22nd 1992 at about 10:15am an attempt was made to launch the AUSROC II Amateur Rocket at the Woomera Instrumented Range. A series of malfunctions occurred which resulted in a failure to launch, and subsequently led to an explosion and total destruction of the vehicle. At 7:00am the 3 hour Flight Firing Sequence commenced. The helium pressure bottle was pressurised to 20MPa and the launcher elevated. The kerosene tank was then filled. A dry nitrogen supply was connected to the LOX tank and the LOX valve opened. The LOX system was then purged for 5 minutes to remove any moisture from the LOX feed system especially the LOX ball valve. The lower valve fairing was inspected visually for any signs of kerosene leakage, and then sealed. At T-30'00", LOX fuelling commenced, and was completed 8 minutes later. It was observed that only a light frost formed on the tank walls when full. At this point kerosene was discovered to be dripping from the base of the rocket. The amount of leakage was assessed to be insignificant, and a decision to continue with the launch was made. At T-15'00" the Final Arm & Launch Sequence began. The ignition circuit was connected and all personnel were cleared from the launcher. At T-2'00" the automatic launch sequence was initiated. Forty seconds later at T-1'20" an ABORT was called. The picture from the onboard camera had suddenly deteriorated. The countdown was held while the problem was discussed, 10 minutes later the automatic launch sequence was restarted at the T-2'00" mark. At T-5s, the electric match fired, and the ignition flare ignited successfully. At T-3s the helium valve opened pressurising both propellant tanks. At T-0.25s the kerosene valve opened (as the kerosene takes about 250ms to travel through the regenerative cooling passage of the motor). At T=0s the LOX valve was actuated, but failed to open fully, resulting in insufficient thrust to lift the vehicle. An attempt to abort the launch was made at T+2s, but the massive kerosene plume had burnt through nearby ground pneumatic lines preventing the abort system from closing the propellant valves. At the same time a crackling or popping sound could be heard. Eventually, at around T+10s, the more characteristic "thrusting" sound developed and the plume became much brighter indicating that some oxygen was present in the chamber. Kerosene continued to be expelled under pressure until T+15s. At around T+20s the electronics umbilical was also destroyed preventing switch off of the payload. With the payload control lines cut, the payload's timer, thinking the rocket had left the launcher, started a 55s countdown to deploy the recovery mechanism. A small fire could be seen at the bottom of the motor, the remaining kerosene dribbling from the rocket burning brightly in oxygen. Kerosene on the ground and around the launcher also continued to burn with a much redder flame. From the onboard camera, smoke could now be seen streaming from the upper valve fairing. At T+1'16" the payload fired the nose separation pins, and then the nose push rod, 2 seconds later. The nose cone popped off to one side, and fell to the ground. At T+1'25" the payload failed, and all telemetry except for the video was lost. At T+1'40" the video transmitter stopped. The flame at the bottom of the motor continued to burn brightly. The fire around the launcher eventually went out about 1 minute later. At T+3'45" a mixture of kerosene and oxygen exploded in the kerosene tank, rupturing the tanks cable duct. The expanding gases tore out both the lower valve, and intertank fairing hatches, and then sheared the bolts fixing the intertank fairing to the LOX tank. The LOX feed line was severed at the LOX tank boss, and the rocket was blown in half. The remaining LOX pressure was sufficient to lift the top half of the rocket off the launcher rail, and propel it through the air and then along the ground for some tens of metres. After a 30 minute cool-off time and careful examination of the wreckage from the periscope in EC2, the operations manager and range safety officer proceeded to the launcher area to make the area safe. Mains power was removed from the area, and the pyrotechnic cutters associated with the main parachute were disarmed. The various pieces of wreckage were gathered together and brought back to Test Shop 1 for examination. The upper portion of the rocket was severely dented, and disassembly was not possible on the day. Most of the fittings in the lower valve and intertank fairings were either missing or very badly burnt. The engine however was removable and it was discovered that the LOX valve had indeed opened by about 10 degrees. The immediate conclusion, reported by most of the media on the day, was that the LOX valve had frozen shut, possibly due to the extended countdown. Eventually it was decided that this was unlikely considering the low humidity on the day and the fact that the dry nitrogen purge should have left nothing to freeze within the LOX valve. The preferred explanation was that one of the pneumatic lines, probably already burning due to the kerosene leak, had burnt through, just as the LOX valve was opening [1]. The remains of the rocket were shipped back to Salisbury to be fully dismantled. The motor, and the remains of plumbing from the lower valve fairing were brought back to Melbourne for inspection. II. FAILURE ANALYSIS ~~~~~~~~~~~~~~~~~~~~~ "The price one pays for pursuing any profession, or calling, is an intimate knowledge of its ugly side." James Baldwin As described above, there were in fact several malfunctions, some of these prevented the launch of the rocket, some contributed to the subsequent destruction of the rocket, and some were simply embarrassing. The major failures which will be discussed are; - sudden deterioration of the onboard camera picture, - the LOX valve failing to open, - kerosene seen dripping from the base of the rocket, - the internal fire - the explosion in the kerosene tank, - failure of the abort sequence to close the propellant valves, or disable the payload, - lack of concrete data with which to analysis the failure. A. Onboard Camera Picture Failure The sudden deterioration of the video picture took the form of saturated white horizontal bars forming about bright portions of the picture. During the launch, these bars were present to an extent, but not enough to really detract from the overall picture. However at the exact time the payload was switched to internal power, these bars suddenly swamped around 30% of the picture. Causing the telemetry personnel to call a hold. The horizontal bars were caused by solid state regulators in the actual camera shutting down (thermal limiting) after overheating. The camera had been connected directly to the rocket's unregulated power supply which is nominally 14V, a little higher than the camera's nominal operating voltage of 12V. The condition became drastically worse when the payload was switched to internal power because the lithium battery pack used to power the payload, was capable of supplying, initially, around 17V. The same effect was repeated in Melbourne, after fire damage to camera had been repaired. The camera was connect to a 14V power supply and allowed to operate for some time (around 20 minutes) until the horizontal white bars developed, then the power supply was raised to 16V, and a very similar effect was observed. The fault could have been avoided had the camera been connected to a regulated power source. In fact a 12V regulator was provided for the camera on the rocket's power supply, but this output had simply not been used. B. LOX Valve Failure The most obvious and vexing question of course, is the reason for the LOX valve failure. The original explanation, that a pneumatic line had burnt through at exactly the right moment seemed a little unlikely. At the last static firing, valve position sensors showed that the time taken for the LOX valve to opens is in the order of around 60ms [2], so for the LOX valve actuator to have moved 11% requires failure within a window milliseconds wide, an unlikely event indeed. Thus a reason which inherently moves the valve a small amount would be infinitely preferable to one which relies on split-second bad luck. Possible reasons investigated included; a) an electrical failure due to ; - an umbilical being disconnected, - an electronic failure in the Launch Sequence Controller (LSC) or it's power supply. b) a pneumatic failure due to; - a loss of pressure to the actuators due to a breach in the ON side pneumatics, a 1MPa regulator failure, or a pneumatic (Legris push fitting) fitting failure, - an electrical or physical failure in the pilot solenoid, - a failure in the vane actuator. c) mechanical failure due to; - the valve seizing due to mechanical distortion from cryogenic temperatures, - the ball freezing to the valve seat due to moisture being present, - the valve stem or perhaps valve sensors jammed due to ice build up, The majority of these possibilities were rejected simply because they did not satisfy the split-second timing problem. An electrical failure was discounted as the LSC's indicator lights showed that appropriate signals were being sent to the pilot solenoid valves. The LSC was tested later and proved to be fully functional. An ON side pneumatic line failure was seriously considered as a possibility. The kerosene leak in the lower valve fairing would have dribbled kerosene onto the pneumatic lines leading to the rocket. These lines would have ignited with the flare, severely weakening them. Conceivably then the kerosene plume exiting from the motor could have burnt through the lines then, as the timing was chosen such that the kerosene and LOX exit almost simultaneously. However high speed film shows no sign of the lines burning beforehand, and the kerosene plume does not exit the rocket motor for about 0.5s. It was also suggested that the kerosene leak may have lubricated one of numerous pneumatic couplings allowing a line to blow off. This was discounted by collecting all the push-fittings and checking that a piece of tubing was still firmly inside the fitting. A failure with the pilot solenoid was rejected mainly due to the timing reasons mentioned. Unfortunately, the solenoid was very badly damaged making it difficult to prove beyond doubt that it was operational. Originally the vane actuator was not even considered as a possible point of failure. However it was mounted directly onto the LOX ball valve, and its mechanism contains two seals which may not operate properly beyond around -20C. Had these seals failed, the expected response would match those observed very well. Thus a seal failure in the vane actuator is a preferable explanation, and is discussed in detail below. The actual LOX ball valve seizing from mechanical distortion was rejected out of hand as the valve is explicitly designed to handle cryogenic fluids. Freezing of the valve stem, or the position sensor was rejected due to the lack of humidity on the day. Even had a layer of ice formed, it is unlikely, given the small surface area, that it would have jammed the vane actuator. In light of the kerosene leak, it was suggested that the whole mechanism may have been frozen in a lump of kerosene ice. However if this was the case, than the valve would not have opened at all. At an earlier static firing (14/3/92) the LOX valve had also failed to open fully. Inspection of the valve afterwards showed that there was trichloroethane present in the LOX valve itself, a remanent from an earlier procedure to remove grease from the LOX feed system. This event caused the addition of a dry nitrogen purge to the launch sequence. Nitrogen is flushed through the LOX feed system, hopefully removing any residual solvents as well as any water vapour present in the tank. This procedure would appear to be successful as the following 3 static firings progressed with out a hitch. For this reason, as a dry nitrogen purge was performed, this theory was discarded. The vane actuator was used to actually turn the LOX ball valve. It was mounted directly to the body by an aluminium mount, and coupled to the valve stem via a slip on coupling. The aluminium block was machined to contact well with both the valve body, and the bottom of the vane actuator. This mount would have formed a reasonable thermal path from the body of the valve to the body of the vane actuator. The seals within the vane actuator are made from polyurethane and have a nominal working temperature range which extends as low as -20C. Beyond this temperature, the seals begin to lose their elasticity. LOX was present at the LOX ball valve for 40 minutes (30 minutes from the start of fuelling, plus another 10 minutes for the hold). With LOX having a temperature of around 90K, in the enclosed environment of the lower valve fairing, it is entirely possible that the vane actuators body temperature could have fallen to unacceptably low temperatures. If this was the case, then the vane would have "frozen" in the closed position. When the pneumatic pressure was applied, the vane would have hesitated and then moved possibly in "stutters". With the seal no longer plastic, the gas may also have burst under the seals delaying the movement even further. With the LOX valve partially open, the plume cuts through the pneumatic lines, while the vane actuator is still stuttering open, some seconds later. This would seem to be the most plausible reason for the LOX valve failure. Hopefully a test can be conducted utilising the Helium valve vane actuator (if it has survived) to confirm this. If this is the case, the abort may have contributed to the failure, as it added 10 minutes to the countdown, extending the time LOX was present at the valve by 33%. C. Kerosene Leak A leak in one of the kerosene valve's body connector seals was detected during final pressure tests the day before launch. As it was a gas leakage at a negligible rate, it was decided to ignore it. On the launch day, after kerosene fuelling, it was observed that no kerosene was leaking from the body connector seal. However after the LOX fuelling, and the sealing of the LOX bleed plug, it was discovered that kerosene was leaking from the bottom of the rocket [1]. The leak in the seal itself was caused simply because the type of body connector seals used in the kerosene valve were in fact once-only seals, that is they deform to form a seal, but once the valve is disassembled they stay deformed, and should be discarded. This was not the case, the seals had been used four or five times already. The leak manifested itself only after the LOX tank had been sealed because of a design fault in the tank pressurising system. The LOX tank is self pressurising in the sense that the LOX is constantly boiling off, so that the pressure rises in the tank once it is sealed. The tank pressurising system was designed assuming that the tank regulators acted as check valves and thus would prevent backflow from a pressurised to tank back into the system [3]. This proved not to be the case. Once the LOX tank was filled, a small amount of oxygen under its own pressure flowed back through the pressurising system and into the kerosene tank. The amount of oxygen would have been very small, however this pressurisation of the kerosene tank was enough to cause the kerosene to leak. The kerosene leak in itself was probably not as major a problem as it sounds. However by dribbling down the umbilical, it supplied a path by which the exhaust plume could ignite the wiring loom inside the lower valve fairing. D. Fire Inside the Rocket A fire inside the lower valve fairing should not have been as major a problem as it was. A tiny volume, mostly sealed at the top, a fire should have quickly suffocated itself. In addition the insulation on the wiring loom was self- extinguishing, that is if lit by a open flame, the insulation does not continue to burn in air once the flame is removed. As was mentioned earlier, the LOX tank self pressurises. For this reason a relief valve is placed at the top of the LOX tank, and set to crack at 4.5MPa. The vent from this relief valve was not piped to the atmosphere, but left within the rocket. During the countdown, the LOX tank would have been slowly venting into the rocket body, and venting furiously during the 15 seconds after T=0s (as can be seen from the onboard camera). This would have provided a very oxygen rich atmosphere within the rocket, allowing the looms to burn up the rocket as far as the payload, eventually destroying it. The amounts of oxygen present can be seen from the severe "weathering" of all the aluminium parts after the fire. E. Kerosene Tank Explosion As mentioned earlier, oxygen was able to bleed back, through the LOX regulator, from the LOX tank to the kerosene tank. After all the kerosene had been expelled, and the helium pressurising gas vented, oxygen bled back through the pressurising system to forming a fuel air mixture within the kerosene tank. When the mixture ratio was right, it ignited from the small kerosene fire seen at the at the base of the motor. The flame travelled back through the motor's cooling passages, and through the injector into the kerosene tank. The residual kerosene may even have been burning inside the kerosene tank for a while before exploding. The explosion ruptured the LOX pipe conduit, at its weld to the top of the kerosene tank boss. The hot gasses then expanding down through the LOX pipe conduit into the lower valve fairing. The lower valve fairing hatch's backing plate was buckled and then blown from the rocket, coming to rest on the launch apron ring road. The upper valve fairing hatch was likewise torn out. Some gas rushed upwards through the pressure line & wiring conduit into the electronics fairing, breaching the camera's case and pushing the main parachute out of it's tube. The bolts holding the intertank to the bottom of the LOX tank boss then sheared, breaking the rocket it two. The upper launch lug broke, and the rocket was thrown to one side. The LOX feed line ripped from it's fitting at the base of the LOX tank, and the thrust produced by the LOX being expelled was sufficient to lift the top half of the rocket, through the air and then along the ground for some distance. The bottom half of the rocket was also torn from the launcher, and fell to the ground nearby, the remaining kerosene visibly burning for a several seconds. The bleed back through the regulator was more complicated than just simple two-way flow through the regulator. It can be shown that had the LOX valve completely failed to open, then the events leading to the explosion could have been avoided (see Appendix A). F. Abort Sequence Originally the rocket was designed with no abort system at all, however at the static firings it was discovered that the existing pneumatics could, with the addition of a few lines, allow the propellant valves to be closed as well as opened. This system used at each of the static firings, and then incorporated into the rocket itself, if only as a convenient method of shutting the valves during tests. The abort system was actuated at about T+2s, but was unable to close the propellant valves because the pneumatic line used to close the valves had already burnt through in the exhaust plume of the rocket. Likewise the payload could not be disabled because the electrical umbilicals also burnt through. The failure of the abort system is the most unacceptable of all the failures as it was thoroughly predictable, and easily avoidable. G. Lack of Data Most of the analysis involved a large degree of speculation because little data of the failure was available. All of the cameras were placed to take rather optimistic "long" shots. So no clear picture of the base of the rocket is available. This was compounded with problems with the payload which resulted in critical data such as the tank pressures, and the valve position sensors being lost. III. SOLUTIONS ~~~~~~~~~~~~~~~ "For every problem there is one solution which is simple, neat, and wrong." H. L. Mencken With "20/20: hindsight, it is easy to propose simple solutions to many of the problems which have already occurred. The real solution is to actively try and find all the possible failures have not occurred and to either prevent them or at least have procedures as to what action to take, when they occurred. As a case in point, the payload could have been disabled in the first 20 seconds after the failure, as the electrical lines where still intact. Although this would not have saved the rocket, at least it would have prevented some media embarrassment. The abort system and payload umbilicals should have been heavily protected from the exhaust plume. An E-flux deflector could be welded to the base of the launcher. The pneumatic lines running to the rocket, as well as those inside should be replaced by stainless or aluminium tubing. The 1MPa pneumatic supply should be moved much further away from launcher, and protected. The electric umbilicals could be connected high up on the rocket so as to be out of harms way. The close valve could be placed inside the rocket so that there is only one pneumatic line leading to the ground. Check valves should be installed after the each propellant tank regulator in order to prevent the bleed back of gases. Both the LOX and kerosene relief valves should be repositioned so that they vent to the atmosphere, not the inside of the rocket. All components used should be carefully studied so that items such as non-reusable seals are replaced, and normal operating conditions are not exceeded. The current LOX valve arrangement could be used with the addition of a thermal insulator such as a plastic or ceramic plate between the body of the vane actuator, and the valve body mount. Extensive testing of each of the possible valve failures should be investigated under realistic conditions (using liquid nitrogen) and worst case data should be obtained. An automatic abort sequence could be added to the LSC in order to cut down response time assuming appropriate telemetry data is available. Better displays of realtime engineering data would also allow better decision making. Finally, more formal procedures, especially launch/abort criterion need to be established beforehand so that these decisions are not made "in the heat of the moment". IV. CONCLUSION ~~~~~~~~~~~~~~~ "You may be disappointed if you fail, but you are doomed if you don't try." Beverley Sills AUSROC II failed to lift off because the LOX valve failed to open fully. The most likely explanation is that the valve only partially opened because the seal inside the LOX valve's vane actuator failed due to prolonged exposure to low temperatures. The sudden deterioration in the live video signal was due to incorrect wiring of the video camera's power supply. The 10 minute hold caused be the camera problem may have contributed to the LOX valve failure. After the LOX valve had failed to open, it should have been possible to save AUSROC II by closing the propellant valves, and disabling the payload. This was not done, as the wires and pneumatic lines associated with the abort system, were not protected in any way, and therefore burnt through in a matter of seconds. Simply shielding the wires and using stainless steel pneumatic lines would have avoided this problem. An automatic abort sequence based on telemetry data would allow the launch to be aborted the instant a valve failure is detected. The explosion which destroyed the vehicle was caused by oxygen flowing backwards under its own pressure, through the LOX regulator into the kerosene tank. Residual kerosene vapour in the kerosene tank mixed with the oxygen to form an explosive mixture. The backflow occurred due to a design fault in the pressurising system, a check valve placed before or after the LOX regulator would prevent the problem. The kerosene leak was caused by a non-reusable seal being reused in the kerosene ball valve. This leak provided an ignition source for the fire inside the rocket, and while it contributed to the destruction of the payload, it probably did not contribute otherwise to the launch failure. Although the wiring looms were self-extinguishing, the placement of the LOX relief valve vent inside the upper valve fairing provided an oxygen rich atmosphere within which they could burn. The relief valve should be placed so that it vents directly to the atmosphere. If AUSROC Projects is to continue another AUSROC II (designated AUSROC II-2) vehicle needs to be built. An opportunity now exists to incorporate all of the changes which had been suggested during the construction of AUSROC II-1, as well as the changes suggested here. The design of AUSROC II was in many ways too "positive". Much thought had been put into each of the systems, but little thought had been allocated to possible failures and their consequences. Obviously, greater testing of each component may have shown up some of these problems earlier. This simply highlights the very limited resources with which the group currently works. The six static firings were in themselves, major system tests, but they were already a major strain on our resources. Hopefully AUSROC II-2 will be able to proceed in an environment where financial and man- hour constraints become secondary to the process of engineering. References [1]AUSROC Projects, AUSROC II Launch Campaign Review, 26 October 1992 [2]A. Cheers, Static Firing Data - 25/4/92 1st Firing, April 1992 [3]M. Blair and P. Kantzos, Design of a Bi-Propellant Liquid Fuelled Rocket, Final Year Project Thesis, Dept. Mechanical Engineering, Monash University, 1989 Author Tzu-Pei Chen Phone: (03) 561 8654, 560 8629ah Ardebil Pty Ltd FAX: (03) 560 5562 6 Kooringa Crescent Pager: (03) 483 4206 Mulgrave VIC 3170 Email: chen@decus.com.au