The justice, who is seen as a conservative, told a meeting of the Federalist Society late on Thursday he was not underplaying the severity of the crisis or criticizing any officials for their response.
But he added: "We have never before seen restrictions as severe, extensive and prolonged as those experienced for most of 2020."
"The COVID crisis has served as sort of a constitutional stress test," he said during his address over a video link for the conservative organization's annual conference https://bit.ly/2H21TvK.
Alito, who was appointed by Republican President George W. Bush in 2006, referred to restrictions on gatherings that had affected religious events.
"Think of worship services! Churches closed on Easter Sunday, synagogues closed for Passover in Yom Kippur", he said.
"It pains me to say this," Alito added, "but in certain quarters, religious liberty is fast becoming a disfavored right."
The justice said freedom of speech was also under threat.
"Although that freedom is falling out of favor in some circles, we need to do whatever we can to prevent it from becoming a second-tier constitutional right," he said.
Alito's remarks on free speech echoed his words from 2016 at the same event when he referred to college campus culture that conservatives say stifles free speech to avoid offending political sensibilities on matters such as gender, race and religion.