The idea of formal systems and Godel's incompleteness theorems comes to mind. 

A formal system is defined by axioms. We try to treat science (and by proxy, the natural world) with an assumption of nature itself being a formal system -> start with axioms and explore the space from there. Axioms and derived theorems then create the boundaries for the environment you're working with. 

Mathematical proofs show you absolute truth in the axiomatic system you are working with. They will also show the absolute falshoods of the system. The space is fractal, endlessly deep, you only can work with the knowledge you have of system you are working with.

The problem is that nature is mind-bogglingly entangled. 

You think CS dependencies are tough? Biological processes seemingly have not only unenumerable dependencies, but dependencies that well eventually circle back to the process itself. At that point, you'd expect a computer to be in deadlock, but it gets worse. There there are not only multiple different clock-cycles, but its a gradient everywhere you go. In biology, it is often said that there are always contradictions to rules being stated and that whatever "rules" we come up with are more guidelines, so we can't even make the assumption that biology and nature is a formal system.

 What are the implications of that? 
A real strangely looped rabbit hole for you 🕳️🔄🐇

From Godel Escher Bach ch 3, a visualization of this process with some defined formal system. Godel's Incompleteness Theorem proved that any formal system will  have statements about it that cannot be proved from within said system. For example, where in the system of language rules do you get meaning from? You're not going to derive semantics from syntax.

https://i.nostr.build/AaXlL.jpg

So if formal systems can't even be fully explicated , what does that imply for non-formal systems? Damned if nature is formal, damned even more if its not.

Now for a sleight shift.
Humans, we live outside many axiomatic systems, but use them to frame our understanding of the world. Ever work on a problem for such a long time, and then stop and do something else? Thats because we aren't bound by said box/axiomatic system and can move outside it. This feature itself allows us to reenter the system from different angles, or just say "Stop, its not worth it."

The capacity for humans to exit systems is a core part that separates us from computers. Wisdom incorperates restraint of application as a consequence from viewing the system wholistically.

The real questions now: What kind of boxes are we living in, be they foundational or self constructed? Are we  agents taxing towards new provable theorems contained in some system we can't exit from? What are the generating functions for this system? Are we the system itself? 

This gives me hope on the unknowable truths and falshoods. Spiritual experience is the experience of The Other, whatever that may be.

Not formally religious, but there is strong appreciation and a wistfulness toward it. 

 This reminds me of the distinction between complicated and complex systems.  A complicated system can be predictably modeled, even if it requires a large model or many steps to do so.  A complex system, on the other hand, is inherently unpredictable and surprising.

Mathematics and other axiomatic systems are very complicated, but still fundamentally predictable.  Arguably physics is complicated rather than complex (though with the observer effect in quantum mechanics, I'm not totally sure about that).  The weird thing is that, even though, say, biology is "just" physics applied to living organisms, it is incredibly complex, unpredictable, and surprising.

Your point about humans being able to shift between systems is particularly salient.  If reality is fundamentally ordered and knowable, we would expect a coherence between various axiomatic systems insofar as they reflect reality, even though no one system has the whole picture.  So we have to remain humble and recognize the extent of our ability to know on our own.

All that said, if we develop and test various axiomatic systems to show they have some bearing on reality, then we ought to be able to triangulate some knowledge about the fundamental reality that those systems describe.  Catholicism recognizes this as "natural revelation," by which, with the aid of reason, we can observe the created world to come to know some things about the Creator, who is ultimately outside of any systems we can devise.

The medieval theologians had this idea of "divine simplicity," which says that God is identical with all of His attributes.  So we might say "God is omnipotent," but in reality, God is not a being who possesses omnipotence, he *is* omnipotence.  And also omnipresence, omniscience, goodness, love, and so on.  Thus God, who is the fundament of reality, is fundamentally simple; He is one thing.  Everything we can say about God is just another way of triangulating on that ultimately simple Being.

Which brings me to a question, based on the end of your post: 

If you are appreciative of and wistful towards religion, what is keeping you from making the plunge and pursuing it further? 

 I'm confident in my compass. This is distinct from  being confident in my beliefs. 
Chemotaxis, or how cells move up a chemical gradient for nutrition, or phototaxis moving towards or away from the light gradient. They don't "know" what is the maximal position to be in, but they move in the direction that is strongest to  help them survive. I argue that this is not entirely mechanistic, that there is a sort of 'taxis of meaning' where agents move towards the highest signal that is meaningful towards them.

That eternal 'search' to something is what I see in Complexity, coupled with being entangled  with the environment. I see all humans doing that in some sense, but what is it? Something arch-typical in mythology and religion is the concept of 'the greatest good'. Every religion has it, I personally don't see myself aligning myself with one completely at every time.

Like this taxis, I use my compass to help me find that which is most meaningful to me, trying to absorb the traits of 'the greatest good' that I find in the world. This can change moment to moment and I'm okay with that. 

I love the lecture series on religion by Jordan Peterson, Eric Weinstein's passion and references to his  Jewish heritage even though he is not religious, Alan Watt's lectures on the Abrahamic Religions, Hinduism and Buddhism. All these are individuals that embody that trait which I want to also take on. But the spiritual practices that I align most with would likely be Taoism, Buddhism,  and particularly the perspectives given by John Vervake and Jordan Hall regarding "The Religion that isn't a Religion".