Irish bottleneck

When I voted to leave the EU in 2016, I thought that leaving would be straightforward. I thought we were halfway out already, we never took the currency and we are not in the Schengen zone. I thought it was an opportunity to close our biggest tax loophole because I knew that some of the countries in the EU were tax havens yet they share a common market with us.

I didn't realize that the EU was acting as a proxy union between North and South Ireland and allowing movement of goods and people between the two states. The problem is on both sides because if we import goods into the UK from outside the EU then the Irish border could become a back door to cheat the tariffs and bring goods into the EU, none of the EU countries would accept that. The other problem is that North and South Ireland want to keep a soft border. I don't know how this problem could be solved without investing large amounts of money. 

The only thing I can think of is to add rebate zone where goods are taxed and the tax is reimbursed straight away and special ID cards are given to North Ireland citizens that would allow them to enter the rest of the UK and also ID cards that would let the Republic of Island citizens only enter North Ireland without a visa.This would cost a lot of money because many staff would be needed and the system itself would cost a lot of money.

The other option would be to let North Ireland have a referendum on if they should stay in the UK. I can't think of other solutions.