We explore how firms grow by adding products. In contrast to most earlier work on the topic, our conceptual and empirical framework allows for separate treatment of product innovation (vertical differentiation) and diversification (horizontal differentiation). The market context is Japan’s cotton spinning industry at the turn of the last century. We find that introducing innovative products outside of the previously feasible set involves removing the “supply-side constraint” by investing in new types of machines and technologies. This process involves a high degree of uncertainty, however, so firms that take steps in this direction tend to first introduce innovative products on experimental basis. We show that conducting such experiments is a key to firm growth. It not only provides opportunities to capture the market in high-end vertically differentiated products when successful, but also facilitates horizontal differentiation of the firm’s products within its previous technical capabilities. In long-term outcomes over 20 years, the right tail of the firm size distribution becomes dominated by firms that were able to expand in both directions: moving first into technologically challenging vertically differentiated products, and then later applying their newly acquired high-end technical competence to horizontal expansion of their product portfolios.