Ancient civilizations did their most impressive work with water. Hydraulic engineering is where the Egyptians, the Persians, the Greeks, the Romans and others devoted the most resources by far and they had some impressive achievements to display.
You will find a lot of information in this Quora blog: Pre-Modern Water Engineering. In there you will learn about qanats, sakia, hypocausts and how natural water flow can be used to do useful work, from pumping water, to producing salt.
I was truly impressed by the sheer scale of the work involved in the construction of qanats, but also with their simplicity and effectiveness as a solution to bring water from the aquifier to the surface. A Persian invention, quite obscure, but also very important.
A schematic of a qanat. Image source: Man Qanat Live By Bread Alone: Water Systems Around the World
One particularly interesting example from an engineering viewpoint is the Tunnel of Eupalinos, a 1036m long aqueduct built in the 6th century BC in Samos and kept in use for 1000 years. The fascinating thing about this is the way that Eupalinos managed to make the two sides of the excavation meet, not to speak of the accuracy of the tunnelling work itself.
Image source and more info on the tunnel: The Eupalinos Tunnel of Samos
Apart from water engineering, ancient weapon technology provides some fascinating feats. Among them, I am impressed by the sheer extravagance of the ultimate siege engine of the antiquity: the Helepolis.
Image source and more info on the siege engine: Ancient Greek war machines: The Helepolis, a fortified wheeled tower
What I found impressive, apart from the size, weight and number of men that manned it, is that it was armored with iron plates on 3 sides and that its enormous wheels could be steered with an enormous capstan, moved by 200 men.
Another cool gadget ancient siege engineering had was Polybolos, a repeating ballista. Medieval Chinese armies made extensive use of the repeating crossbow, aka the "chu-ko-nu". The former operated on a different principle (chain drive) than the latter (lever operated).
Chain drive of the repeating ballista. Image source and more info on the siege engine: Repeating Catapult
This answer also would not be complete without mentioning the Antikythera mechanism.
X-ray imaging of the main part of the Antikythera mechanism. Image source and more info on the mechanism: Advanced Imaging Reveals a Computer 1,500 Years Ahead of Its Time
The astronomical and mathematical knowledge, as well as the engineering genius and the exceptional metalworking craftsmanship that went into making this are simply unbelievable, for something that was truly 1500 years ahead of its time.