We quantify the importance of imperfect competition in the U.S. labor market by estimating the size of rents earned by American firms and workers from ongoing employment relationships. To this end, we construct a matched employer-employee panel data set by combining the universe of U.S. business and worker tax records for the period 2001-2015. Using this panel data, we identify and estimate an equilibrium model of the labor market with two-sided heterogeneity where workers view firms as imperfect substitutes because of heterogeneous preferences over non-wage job characteristics. The model allows us to draw inference about imperfect competition, compensating differentials and rent sharing. We also use the model to quantify the relevance of non-wage job characteristics and imperfect competition for inequality and tax policy, to assess the economic determinants of worker sorting, and to offer a unifying explanation of key empirical features of the U.S. labor market.