I have a vague recollection of this issue. But I have no idea in which version it was corrected, or if there was a workaround.
I would suggest:
1- Try a smaller file (if only large files are an issue, it's a hint to look at MySQL config)
2- Check if tracker attachment storage has an option to store in DB vs filesystem. Whatever the current setting, try the other one.
3- Do a test upgrade on a backup copy. Test an upgrade to latest 3.x, latest 6.x, latest 4.x and latest 5.x
Old releases are available from Download page. Unless the Tiki code has been modified, you will see that the upgrade will be easy. The best is to upgrade to Tiki6LTS.
Once you know which version solves the issue (ex.: 3.x), you can look through the source code to try to find the commit which resolves the issue:
As you offer services, I suggest you get listed on the Consultants list.