U.S. District Judge Richard Young's ruling did not have a stay on it and couples began getting married immediately.
The Indiana attorney general's office said it will ask for a stay while it appeals the ruling to a circuit court, said Bryan Corbin, a spokesman for the attorney general.
Evan Wolfson, president of the gay rights group Freedom to Marry, applauded the Indiana decision.
"Today's decision out of the heartland underscores that America is ready for the Supreme Court to bring an end to marriage discrimination once and for all," he said in a statement.
Couples arrived at the Marion County clerk's office in Indianapolis within 10 minutes of the ruling to get marriage licenses, said Erin Kelley, a spokesperson for Clerk Beth White.
By 6 p.m. EDT, the clerk's office had performed 63 ceremonies and issued 90 marriage licences, Kelley said. The office was going to stay open until 8 p.m. to meet the high demand.
"I lost track of time about 10 minutes after the ruling," Kelley said. "It has been a whirlwind."
In a separate decision Wednesday, a federal appeals court in Utah became the first to back gay marriage.
The U.S. 10th Circuit Court of Appeals upheld lower-court decisions that struck down Utah's ban on same-sex marriage. That ruling sets the stage for an appeal to the U.S. Supreme Court.
Brian Brown, the president of the conservative National Organization for Marriage, said in a statement that he expects the decision to be reversed by the Supreme Court.
"The elected representatives of the people of Indiana have decided, for good and proper reasons, to define marriage as the union of one man and one woman," he said. "It is judicial activism for a single judge to substitute his own views on marriage for the considered opinion of the people's representatives."