Football has taken many interesting twists and turns, some large and others small, including being the source of varsity letters. You might think letters were always letters, and they were awarded in the past just like they are today, but that was not the case.
Varsity letters originated informally when teams began allowing players to keep their jerseys after playing in key games or starting for most of a season. However, some people who had not earned the right to wear a letter sweater began wearing them around campuses, and when "minor sport" athletes began earning letters as well, there were efforts to distinguish minor from major letters.
In the 1800s, football, baseball, track, and crew were the major sports. Some schools considered hockey a major sport, and basketball became one later. Still, whatever the mix on each campus, many formalized which people could wear school or team-specific logos.
For example, in the spring of 1896, Wesleyan formalized who could wear different sweater and hat colors that indicated the wearer earned a letter in a given sport.