The pact, announced jointly by Kerry and Karzai late on Saturday after two days of talks in the capital, Kabul, would keep some U.S. forces in Afghanistan after the withdrawal of most foreign troops by the end of next year.
The draft includes a U.S. demand to retain legal jurisdiction over its troops who stay on in Afghanistan, which would give them immunity from Afghan law.
Karzai opposes that and said the question could not be decided by his government. Instead, a Loya Jirga, or an assembly of elders, leaders and other influential people, will consider the demand and decide whether to accept it.
The United States is insisting it cannot agree to a deal unless it is granted the right to try in the United States its citizens who break the law in Afghanistan.
"We need to say that if the issue of jurisdiction cannot be resolved, then unfortunately there cannot be a bilateral security agreement," Kerry told a news conference.
U.S. officials said they wanted the pact finalised by the end of October and Kerry's visit was seen as a last-ditch effort to push it through before the deadline.
A senior U.S. administration official said the sides had agreed on language in the draft deal that covers the issue of immunity and "that can be put to his Loya Jirga for their consideration".
Hundreds of people, and at times even more than 1,000, attend a Loya Jirga, which makes it difficult to predict how deliberations will unfold.
However, in the past, they have voted in favour of keeping a U.S. presence inAfghanistan and Western diplomats are optimistic the assembly will pass the deal - as long Karzai remains in favour of it.
"The Loya Jirga here basically follows the advice of the president, and I think he will (agree to the pact)," said one senior diplomat.
Karzai said the assembly would be held within a month.
Karzai said the talks had focused on protecting Afghan sovereignty and that major differences had been resolved, including a U.S. request to run independent counter-terrorism missions on Afghan territory.
Such operations carried out by the United States have long infuriated the Afghan president, who had been demanding that the United States agrees to share intelligence instead.