Can you wear only Brief in public?

Under the proper conditions, yes. For example, a male model for Calven Klein is permitted to showcase the briefs in publica as a marketing strategy (e.g., a Fashion Show). Then again, you may want to consider the social environment as well (e.g., stuck in the middle of the Amazon jungle with the Indigenous people who also cover their loins with similar clothing apparels).