This argument for open borders assumes that inequality is fixable.
Those who put forward this argument believe that every inequality that emerges from luck needs to be fixed.
For others, the real flaw in this argument is that its conclusion does not seem to follow from its ethical foundations. There are other means for prosperous nations to meet provide justice and meet their ethical obligations, not just open borders.
Foreign aid and investment is another form of justice that helps better the contexts of other countries
This is how wealthy individuals and countries can provide justice, without opening borders to all immigrants.
For example, as long as Jeff Bezos gives a portion of his wealth to the less fortunate, he has every right to keep others out of his home. If the domestic environment is like this, then why should the international one be any different?