The robbery in the upscale Ginza district was an unusual attempted heist in a country with famously low levels of crime.
Three people entered the shop with their faces covered and armed with crowbars which they used to destroy glass cases and snatch more than 30 items worth over 100 million yen, local media said.
Contacted by AFP, police were not immediately available for comment.
Public broadcaster NHK said four people were taken into police custody, and another suspected member of the gang was still at large.
The robbery left the shop's display windows shattered, with a box bearing the logo "Rolex" visible at the scene.
While Japan is known for low crime levels, and even valuable lost items are often returned to owners, Ginza's fancy boutiques have been targeted before.
In 2014, thieves made off with a diamond ring worth over $300,000 in a Christmas Eve robbery.
And in 2007, the "Pink Panther" gang stole items worth more than $3.5 million from a jewellery shop in the district.
