Mary and Joseph were business owners not poor. They traveled to Bethlehem to pay taxes. Mary was about to give birth, so her husband wanted to rent a room at the Inn.
But, because it was tax season, there was no room at the Inn. They sought shelter in the ”Manger” until the birth. This was a place the travelers kept their animals while staying at the Inn. Today it would be the hotel parking garage. Then they went home (probably after taking care of their taxes.)
The ”Wise Men” visited (the toddler) years later at their home. Then Mary and Joseph took Jesus to Egypt to avoid the Government.
Rev. Clarence A. Mason (aka Mason Weaver)