Alicia Keys & Jay Z Perform in the Concrete Jungle

Alicia Keys and Jay Z busted out NYC's anthem in Times Square.

The singer held a surprise concert where a bunch of other stars also showed ... including Jay Z. Check out the vid ... they rocked "Empire State of Mind" and the crowd went nuts when Hova appeared.

Alicia also covered "Imagine" in honor of John Lennon's birthday.

#Classy