Many were inside the artwork, which consists of connected rooms, when it lifted 30ft into the air at Riverside Park, Chester-le-Street, County Durham.
The Dreamspace sculpture is thought to have drifted for about 40m.
A 38-year-old woman from the town and a 68-year-old woman from Seaham died. A three-year-old girl was seriously injured and was flown to hospital.
The sculpture - called Dreamspace - is thought to have drifted for up to 40m before catching on a CCTV camera post and coming down near a children's playground.
About 500 people were in the vicinity of the sculpture, which is half the size of a football pitch, when it came loose.
Fire officers spent much of their time trying to account for people missing inside the structure. Six ambulances and two air ambulances were sent to the scene.
Durham Fire Brigade spokesman John Robson said: "There's been a large inflatable structure, which included walls and floors and obviously the top, which has somehow lifted off the ground, and moved between 30 and 40m out of its pitched area.
By evening, the remains of the structure were strewn across the grounds of Riverside Park. The red, yellow, and blue polythene was shredded and scattered over a wide area.
