In 1889, John Hornor Wisner created his country estate and built the colonial Revival residence, presently the administrative center known as Wisner House. For an overall landscape plan, he commissioned Calvert Vaux, a partner of Frederick Law Olmsted in the creation of Central Park's sweeping "greensward " look. Mrs. Wisner planted the first clusters of daffodils that are now a major Arboretum attraction in April. In 1916, the new owners, Mr. and Mrs. Richard Reeves, expanded the daffodil collection and, like the Wisners, were guided by prominent landscape architects Ellen Biddle Shipman in 1924 and Carl F. Pilat in 1924-1925. The 1925 rose garden with connecting rock-pool garden was an implementation of "garden room" landscaping, an emerging trend in country estates. Elegant stone steps cascading from the house were completed by Italian masons and, today as then, represent mastery in stonework. An original Shipman landscape plan and a hand-colored image of Mrs. Reed's azalea garden in the 1930s are part of the Arboretum's archives.
In 1968, the Charles L. Reed family became the last private owners, adding the patterned herb garden, opening woodland trails and upholding the property's design heritage. In 1974, local citizens, including the Reed family, raised most of the purchase price to preserve the estate as an arboretum. Additional funds were also provided by the City of Summit, which became property owner. At that time George Morrison Hubbard, Jr. purchased the adjacent field from Charlie Reed which provided the final funds necessary to establish the Arboretum. He subsequently donated the field and adjacent land to the Arboretum in 2007, bringing the property nearer to its historic boundaries.