Jessie Fleming scored in the 48th minute to give Canada the lead, turning in a cross from Nichelle Prince, who added the second with 11 minutes left.
But who were the biggest winners and losers from the game? It was also the first for Prince who is playing in her first World Cup.
Only starting due to injury concerns over Allysha Chapman, Riviere was a wonderful replacement at right-back.
The Canadians finished with 70 percent possession, completing 523 passes to New Zealand's 145. It was her 19th career World Cup outing - and 284th senior game for Canada.
Prior to Saturday, Canada had never advanced past the group stage in a World Cup tournament held in Europe. "There was more aggressiveness in the penalty box as well". Canada vs New Zealand match will be telecast on Sony ESPN and Sony ESPN HD TV channels. It was Prince's 11th goal for Canada and first at a World Cup.
But that resilient defence would not last too long after half-time, thanks to Jessie Fleming. Victory for either the Netherlands or Canada will secure a spot in the round of 16, while Cameroon and New Zealand are fighting to remain in contention.
Led by Tom Sermanni, former coach of Sweden and the United States, New Zealand was hurt in the first half when defender CJ Bott was injured and had to be subbed out.
The killer touch came shortly after half-time, however.
Prince Harry, Meghan Markle Share 1st Photo of Baby Archie's Face
Last month, the Duchess of Sussex - who is American - celebrated her first US Mother's Day by sharing a picture of Archie's feet . The baby hadn't yet been born when the United Kingdom had its Mother's Day this year. "Happy Father's Day! " reads the caption.
Donald Trump slammed for sharing Katie Hopkins tweet
Khan was guilty of "pathetic" behaviour and in July previous year he claimed the Mayor had "done a very bad job on terrorism ". He said Mr Khan was "rightly supporting the police to do their job while Katie Hopkins spreads hateful and divisive rhetoric".
Huawei ‘cautious’ with its foldable device following Galaxy Fold troubles
We reached out to Huawei for more clarification but have so far only been able to confirm that the September delay is real. In addition to an executive order barring the purchase of Huawei's telecom equipment, the U.S.
England, France, Germany and Italy have also booked their ticket to the knockout stage, making Canada the first non-European side to advance.
Subscribe to Squawka's Youtube channel here. With that in mind, New Zealand is going to need to play a more open style of football than they did in their opening game loss.
Canada had an unheard of 73% of the possession in the first half and their best chance fell to Christine Sinclair off a corner kick in the 18th minute.
Sinclair nearly made it 2-0 soon after, but her redirect from in front went high. After good wing play, the ball came to Sinclair on the penalty spot with the 36-year-old nearly certain to score.
There was still time for Sinclair to hit the woodwork once more before the game was up, heading against the post - though Prince tucked in the rebound. Her understanding with defender Riviere was on another level and made it appear as though the pair play together week in, week out. Prince also tallied an assist on Fleming's opening goal.
Prince came close in the 71st minute when she got to a cross by Sinclair and headed it toward the top corner of the net, but Nayler made an outstanding diving save.
The result sees Canada into a two-way tie with Netherlands atop Group E of the 2019 FIFA Women's World Cup, with the Dutch holding a slight advantage in goals scored. And the truth is, they didn't really look like changing that against Canada, being completely overrun by a vastly superior side.
After dispatching No. 46 Cameroon 1-0 and 19th-ranked New Zealand, fifth-ranked Canada now heads 600 kilometres north to Reims where it will face the eighth-ranked Netherlands on Thursday to decide who tops Group E. Will the Football Ferns finally get their first World Cup win, or end a fifth campaign without one?