Jessie Fleming scored in the 48th minute to give Canada the lead, turning in a cross from Nichelle Prince, who added the second with 11 minutes left.
But who were the biggest winners and losers from the game? It was also the first for Prince who is playing in her first World Cup.
Only starting due to injury concerns over Allysha Chapman, Riviere was a wonderful replacement at right-back.
The Canadians finished with 70 percent possession, completing 523 passes to New Zealand's 145. It was her 19th career World Cup outing - and 284th senior game for Canada.
Prior to Saturday, Canada had never advanced past the group stage in a World Cup tournament held in Europe. "There was more aggressiveness in the penalty box as well". Canada vs New Zealand match will be telecast on Sony ESPN and Sony ESPN HD TV channels. It was Prince's 11th goal for Canada and first at a World Cup.
But that resilient defence would not last too long after half-time, thanks to Jessie Fleming. Victory for either the Netherlands or Canada will secure a spot in the round of 16, while Cameroon and New Zealand are fighting to remain in contention.
Led by Tom Sermanni, former coach of Sweden and the United States, New Zealand was hurt in the first half when defender CJ Bott was injured and had to be subbed out.
The killer touch came shortly after half-time, however.
India's retaliatory tariffs on 28 American goods take effect
US-India bilateral alternate used to be rate $142bn (£111bn) in 2018, a sevenfold extend since 2001, in step with US figures. The tariffs come just days ahead of a scheduled meeting between Trump and Prime Minister Narendra Modi at the G-20 summit.
Sarri Leaves Chelsea And Takes Over at Juventus
The Europa League win last month was Sarri's first major trophy as a coach, but he will be expected to deliver far more in Turin. Chelsea will reportedly turn their attention to Derby County in an effort to appoint Frank Lampard as their new manager.
Petsmart's Chewy IPO soars 59% in stock market debut
To grow, Chewy plans to create more in-house pet food brands, expand its year-old animal pharmacy and venture beyond the U.S. The offering follows a heated dispute between PetSmart and its creditors over the transfer of Chewy assets ahead of the IPO.
England, France, Germany and Italy have also booked their ticket to the knockout stage, making Canada the first non-European side to advance.
Subscribe to Squawka's Youtube channel here. With that in mind, New Zealand is going to need to play a more open style of football than they did in their opening game loss.
Canada had an unheard of 73% of the possession in the first half and their best chance fell to Christine Sinclair off a corner kick in the 18th minute.
Sinclair nearly made it 2-0 soon after, but her redirect from in front went high. After good wing play, the ball came to Sinclair on the penalty spot with the 36-year-old nearly certain to score.
There was still time for Sinclair to hit the woodwork once more before the game was up, heading against the post - though Prince tucked in the rebound. Her understanding with defender Riviere was on another level and made it appear as though the pair play together week in, week out. Prince also tallied an assist on Fleming's opening goal.
Prince came close in the 71st minute when she got to a cross by Sinclair and headed it toward the top corner of the net, but Nayler made an outstanding diving save.
The result sees Canada into a two-way tie with Netherlands atop Group E of the 2019 FIFA Women's World Cup, with the Dutch holding a slight advantage in goals scored. And the truth is, they didn't really look like changing that against Canada, being completely overrun by a vastly superior side.
After dispatching No. 46 Cameroon 1-0 and 19th-ranked New Zealand, fifth-ranked Canada now heads 600 kilometres north to Reims where it will face the eighth-ranked Netherlands on Thursday to decide who tops Group E. Will the Football Ferns finally get their first World Cup win, or end a fifth campaign without one?