It usually applies to structures roads, schools, hospitals, airports, buildings and so on. The level of development of a country is the amount of useful infrastructure it has.
Useful infrastructure allows a higher productivity - people can produce more if there are good roads, good schools, hospitals and so on, and the standard of living in countries with a high level of infrastructure is also high.
Economists like to classify countries into the 'developing' and 'developed' world. This is really a measure of its infrastructure.