The opposition is also ahead in the contest for mayor of the largest city, Istanbul, the election commission says.
Nationally, the president's AKP-led alliance has won more than 51% of the vote in the municipal elections.
The vote, see as a verdict on Mr Erdogan's rule, has been taking place during an economic downturn.
The currency, the lira, has been losing value recently and the economy went into recession in the last three months of 2018.
The president had previously said the poll was about the "survival" of the country and his party.
Commenting on the results in a speech on Sunday, he said: "If there are any shortcomings, it is our duty to correct them."
More than 57 million people in the country were registered to vote for mayors and councillors. State-run Anadolu news agency said turnout was high at just under 85%.
What are the results?
The secularist Republican People's Party (CHP) candidate Mansur Yavas won in Ankara.
Istanbul has been in the hands of parties linked to Mr Erdogan since 1994 when he was elected the city's mayor.
The election commission said the CHP's Ekrem Imamoglu was leading there by only 28,000 votes, but added that the results of more than 80 ballot boxes were being challenged.
Both CHP and Mr Erdogan's AKP - or Justice and Development Party - had been claiming victory in the city.
The AKP had been saying its candidate, former Prime Minister Binali Yildirim, was ahead by 4,000 votes. Ruling party officials said they would challenge thousands of ballots in Istanbul and Ankara.
The CHP also said it had held Izmir, Turkey's third largest city.
What has the reaction been?
"The people have voted in favour of democracy. They have chosen democracy," CHP leader Kemal Kilicdaroglu said.
In his speech on Sunday, Mr Erdogan vowed to focus his leadership on the Turkish economy ahead of national elections scheduled to take place in 2023.
Prominent journalist Rusen Cakir said the vote was "as historic as that of 1994", referring to the year Mr Erdogan was elected mayor of Istanbul.
"It is a declaration that a page that was opened 25 years ago is being turned," he said.
Analysis by Mark Lowen, BBC Turkey Correspondent
President Erdogan had painted this election as a matter of survival. He's now been dealt an agonising blow.
For the first time in a quarter of a century, his party has lost Turkey's capital Ankara.
And in the economic powerhouse of Istanbul, there's a hair's breadth between the governing AK Party and the opposition.
As the official tally showed fewer than 3,000 votes between them in this city of 18 million, both said they'd won.
But then the count stopped, with more than 1% of ballot boxes still unopened: a tactic, says the opposition, to steal victory.
This could be a watershed moment for Turkey's powerful, polarising president: when an opposition long seen as moribund finally feels he's beatable.
How was the campaign?
This was the first municipal vote since Mr Erdogan assumed sweeping executive powers through last year's presidential election.
The AKP, with its roots in political Islam, has won every election since coming to power in 2002.
With most media either pro-government or controlled by Mr Erdogan's supporters, critics believe opposition parties campaigned at a disadvantage.
The opposition pro-Kurdish Peoples' Democratic Party (HDP) said the elections were unfair and refused to put forward candidates in several cities.
Some of its leaders have been jailed on terrorism charges, accusations they reject.
Mr Erdogan's rallies dominated TV coverage. At one on Saturday, the president sought to reassure voters and the party's usually conservative supporters that everything was under control.
"I am the boss of the economy right now as president of this country," he said, also blaming the West and particularly the US for its financial turbulence.
The president was criticised for repeatedly showing footage from the recent terrorism attack in New Zealand, in which a self-declared white supremacist live-streamed himself killing 50 people at two Christchurch mosques.