The application of heuristic algorithms with life cycle assessment (LCA) is a promising approach to efficiently optimize the building and minimize the environmental impact even in a very large design space. In this paper, a modular parametric optimization framework combining advanced building modelling, LCA, energy calculation and single- and multi-objective environmental optimization was developed and applied to a multi-apartment house in Hungary. The variables comprise geometry, envelope, fixtures, as well as the heating energy source. The objective is the life cycle environmental impact expressed in terms of the six most common LCA indicators. The results show that a relatively compact shape, large windows to south equipped with shading and very high levels of insulation are optimal from an environmental perspective. It was also proven that focusing only on the embodied or operational impact leads to a suboptimal solution, and the choice of the energy source for the operation of the building has a significant influence on the achievable optimum. Through the optimization, significant environmental savings of 60–80% were achieved compared to the initial design options showing the potential for the application of such methods in architectural design.