According to local police, the 26-year-old Austrian was killed on Sunday just before midnight after he attacked a security guard while trying to enter the ambassador's residence in Vienna's Hietzing district.
Austrian police spokesman Harald Moeser said that the assailant "died on the spot" after the guard shot him four times, after pepper spray failed to stop him.
The crime scene has been closed off by the police, who said the motive of the attacker is still unclear.
The Austrian police added that a security guard was injured in the attack and transported to hospital for treatment.
The attack came several hours after Iranian Ambassador to London Hamid Baeidinejad announced that he had been threatened with death by a member of an extremist religious group, which recently carried out an attack on the Islamic Republic's embassy in London.
A number of attackers scaled the wall of Iran's Embassy building in London on Friday and took down the Iranian flag.
According to the Islamic Republic News Agency, IRNA, British police did nothing to protect the compound during the attack.
Iran on Saturday summoned the British ambassador to Tehran over the attack on its diplomatic mission in London.
