The couple announced the arrival of son Jack Blues Bieber on Aug. 23

Love is in the air for the Biebers and their family of three!

Almost a month after Justin and Hailey Bieber announced the 𝐛𝐢𝐫𝐭𝐡 of their first 𝘤𝘩𝘪𝘭𝘥, a source tells PEOPLE that the couple appears “more in love” than ever now that they have son Jack Blues Bieber in their lives.

“They had a quiet wedding anniversary celebration,” the insider says, referring to Justin and Hailey’s recent 6th anniversary of their September 2018 New York City courthouse wedding.

“They seem even more in love since the 𝑏𝑎𝑏𝑦 arrived,” the source adds.

The source also shares that the new mom and Rhode founder is “feeling great,” and that her husband is “encouraging her to catch up with friends.” The new mom has “been out a few times without the 𝑏𝑎𝑏𝑦,” the insider says.

Meanwhile, the source also notes that new dad Justin has “been in a happy bubble since 𝑏𝑎𝑏𝑦 Jack was 𝐛𝐨𝐫𝐧,” adding that the 30-year-old Grammy winner “wants to focus on being a great dad and husband.”

The proud parents announced the arrival of their son on Aug. 23 by sharing the news on Instagram, alongside an image of the new𝐛𝐨𝐫𝐧’s foot

His name Jack also continues a family tradition for the Biebers — as the 𝑏𝑎𝑏𝑦 shares the initials ‘JB’ with his dad, his grandfather (Jeremy Bieber) and some of Justin’s siblings.

Since Jack’s 𝐛𝐢𝐫𝐭𝐡, Justin and Hailey, 27, have provided just a few details on parenthood via social media, such as when Justin shared a peek at his cute “papa bear” coffee mug on Sept. 3 and Hailey also shared a glimpse of a “mom” ring with her followers the same day.

On Sept. 13, Hailey posted a touching tribute on her Instagram Stories in commemoration of the pair’s latest wedding anniversary. She documented what appeared to be an at-home celebration by sharing images featuring balloons, candles and an adorable instant camera picture. The photo itself showed her and Justin sharing a kiss while posing in front of candles.

Other members of the Bieber family have shared insight into life now with 𝑏𝑎𝑏𝑦 Jack, as Hailey’s dad Stephen Baldwin described his “cute” new grandson while attending the People + Chain 50th anniversary party in Los Angeles on Sept. 13.

“I don’t want to say too much. He’s unbelievably cute,” the actor said at the time, adding that “the world will see him soon.”

In late August, a source also told PEOPLE that the “Baby” singer was “already a great dad” and that the “pregnancy was something that they very much wished and prayed for.”

“The day they found out that Hailey was pregnant, was the best ever for Justin,” the source said, adding that he was “over the moon with excitement.”

“It was a big celebration for them,” the insider said.