West Ham United's Declan Rice gave the ball away badly for Berisha's second while Manchester United's Harry Maguire clumsily failed to clear before hauling down Vedat Muriqi, who scored from the spot.
But before 20 minutes of the match had passed, Kosovo's illustrious opponents were leading.
Keane atoned for his error by setting up Raheem Sterling, who restored parity with a close range header.
A great game to watch but England players have to be self-critical.
And it proved to be their undoing once again in the opening minute. Sterling broke in to the space left behind and found his partner in attack Harry Kane.
It's possibly Kosovo's biggest match and if they can't handle it there's every chance of a red in the match at 5-1 with Sky Bet.
England manager Gareth Southgate praised Rashford's role coming in from the inside left position in winning a penalty against Bulgaria and is encouraging the player to use his speed, rather than move into areas of the pitch where he is less effective.
And they cut the deficit to two 10 minutes into the second half.
Hailee Steinfeld Offered Kate Bishop Role in 'Hawkeye' Series!
The Marvel Cinematic Universe is set to create a very unique form of connected media when the Disney + Streaming Service arrives. Plus, she just recently starred on a TV show for the first time as Emily Dickinson in the upcoming Apple TV series Dickinson .
AppleTV+ details announced: New streaming service will cost just $4.99
The camera on iPhone 11 features night mode capabilities which will come on automatically to brighten photos and reduce noise. Barely hours before the launch of the all-new iPhones , a massive leak has apparently revealed their pricing.
House lays out its impeachment probe
The committee has also invited two other witnesses mentioned in the report , former White House aides Rick Dearborn and Rob Porter.
Fidan Aliti went down after a challenge from Trent Alexander-Arnold, but England played on.
The third goal was booed by the Kosovo fans but showed that England finally have a bit of steel.
Kosovo's players and coaches were furious but the goal stood.
Sterling hit the post and then had a shot deflected wide - nothing was going right for England, flaming Kosovo's belief there was still something for them in this game. "So, that in itself is a stat that should make people take notice". I know the tickets have sold out really quickly, so we're looking forward to a brilliant atmosphere tomorrow night.
Harry Kane will surely break Wayne Rooney's England record of 53 goals.
But England battled back well and found themselves level just seven minutes later. As the clock ticked into injury time, chants of "Edhe nje!" "I think you have to maintain the highest possible quality".
"I think they possibly don't, because they sometimes don't view themselves as others do".
England captain Kane is enjoying a more prolific start to the season, the Tottenham talisman having netted six goals in five competitive appearances for club and country. Kosovo would continue to attack.