AFGE was founded on October 17, 1932, by local unions loyal to the American Federation of Labor (AFL) and left the National Federation of Federal Employees (NFFE) when that union became independent of the AFL (NFFE has in recent years become part of the IAMAW, which is affiliated with the AFL-CIO).

AFGE is a Federation of Local unions, with each Local maintaining autonomy through operating under local constitutions that comply with the AFGE National constitution ratified originally during its founding in 1932.

Federal employees' right to organize and bargain binding labor contracts was established in law by the Civil Service Reform Act of 1978, which AFGE helped to draft, and which states that collective bargaining in the federal sector is in the public interest while also barring the right to strike.

AFGE has played a crucial role in the struggle for women's rights and civil rights in the federal sector, and was one of the first unions to establish a Women's Department and a Fair Practices Department, with the officer over those Departments holding a seat on the NEC and with Women's and Fair Practices Coordinators elected in each AFGE District since the early 1970s.

AFGE's December 2009 court suits stopped aspects of the George W. Bush Administration's “National Security Personnel System” (for DOD) and MAXHR (for DHS), and AFGE also won recent changes to law that make the contracting out process more balanced in regard to federal employees' interests. In 2010, the Obama Administration issued an Executive Order for the Federal Government to focus on insourcing Federal jobs rather than outsourcing them overseas or to contractors.

AFGE's motto was established as “To Do For All That Which No One Can Do For Oneself”.

AFGE's original emblem was a shield with the stars and strips and the words “Justice, Fraternity, Progress” and the current emblem is three workers supporting a globe with a map of the United States and the words “Proud to Make America Work”.

In June 2011, AFGE also won the historic largest single nationwide consolidated bargaining unit election of over 44,000 employees of the Transportation Security Administration, part of the Department of Homeland Security. AFGE is working for a change in law which will give them the same collective bargaining rights as other federal employees.