Fig trees often are compared to Israel throughout the prophets. They are even part of the decorations in the temple. There were carvings in the outer partition of the temple, grapes (makes sense), pomegranates (likewise), and figs as well as other signs of the provision of God in the Promised Land.

So when He is casting judgment on the fig tree, one can assume He is talking about the time He is spending in Israel and their rejection of Him. Even the statement of the three years of no fruit seems to point to the length of His ministry thus far and the rejection by the leaders.