Well, it is of course clear that all the weird issues and limitations (not-yet-implemented features) are the result of them starting from scratch with an own implementation...
Any experienced software project manager or designer or programmer would have predicted this beforehand. It always is more difficult in practice than envisioned before.
The reason for the redesign was that in the original BGP the whole daemon was single-threaded and so on the large CCR routers with 36 or 72 cores the BGP handling would use only a single (comparatively slow) core. Many years ago there were lots of complaints about that from users of the large routers that wanted to run several full route BGP feeds, and it was promised that the holy grail of v7 would solve all their worries by making it multithreaded and generally more efficient.
But that was said of so many other unrelated issues and limitations. Everything would be solved in v7. The v7 releases was pushed forward time after time, and many solutions promised for v7 were eventually introduced in v6. Except that new BGP daemon.
When v7 was finally released it appeared to the user much the same as what v6 had become in the meantime, except the routing (route tables, policy rules, BGP, OSPF) which was restructured and rewritten. That is when the trouble started, there were many subtle bugs and each time we had to pray that a new release was around the corner that fixed some of them.
But after an initial start where some things indeed were fixed (mainly in the multiple route table management), unfortunately it appears that focus at MikroTik has shifted away from this issue and on to other features that are new to v7 and maybe requested by more or by more important customers...
At this time, the progress is so slow that I even fear that the responsible developer(s) for this part of RouterOS have left or have been assigned other tasks.
In a week time we will celebrate the first anniversary of the famous reply by mrz: "BFD is currently work in progress." in the v7.1rc2 topic... viewtopic.php?p=877008#p877008
As far as I understand it, at least the function of BFD as it was present in v6 (which was fine for us) is little more than what the extended "netwatch" function can do as well: send/receive some ping-like packet (in this case over UDP) and take action on persistent failure (close the peer connection immediately). I cannot understand why after a year of "work in progress" that still isn't finished, unless actually nobody is really working on that. I even suggested that the guy who extended "netwatch" adds a BFD mode to his code so we can use that. Maybe that can get some manpower assigned to this dire situation.
But indeed, as you mention, there are other issues. A responsible way would have been to keep the v6 BGP as an optional package that could be used instead of the new one, much as is done with wireless. That would at least allow those that need a working BGP and do not care so much about the new implementation to run v7 on production routers. And that becomes even more important because new models cannot run v6 anymore.
I guess the original BGP in v6 was originally based on BIRD but it has been doctored so much that it would not be a 1-day job to compile it on v7, and therefore it was decided not to pursue that. And of course, when v6 BGP would become available as an option, people would request additional features in that, taking away manpower from the v7 BGP implementation. But MikroTik should understand that "we" cannot wait two years for them to finish their new code...