Community discussions

MikroTik App
 
texmeshtexas
Member Candidate
Member Candidate
Topic Author
Posts: 118
Joined: Sat Oct 11, 2008 11:17 pm

Traffic drops with BGP routing

Thu Oct 27, 2022 5:30 pm

I have a strange problem that I wanted to get input on.
I have two edge routers. One is peered to one upstream ISP1 and the other to another ISP2. Both using default route (not full routes)
I want to advertise all prefixes to both ISPs but I get packet drops that cause certain sites to not load right (facebook, youtube, vpn connections).
Both routers are CCR1072 V6.49.7
Here is a diagram to show how things are connected.

Image

This is a simplified diagram as the real network has many more downstream sites/prefixes/etc. But it captures the basic idea and problem.

Anyone have an idea why I get packet drops?
It is site dependent, if traffic happens to come back in the same router that it went out, not problem.
You do not have the required permissions to view the files attached to this post.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 6815
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Traffic drops with BGP routing

Thu Oct 27, 2022 6:05 pm

In case of asymmetric routing make sure that you are not using any of the connection tracking related features.
 
texmeshtexas
Member Candidate
Member Candidate
Topic Author
Posts: 118
Joined: Sat Oct 11, 2008 11:17 pm

Re: Traffic drops with BGP routing

Sat Oct 29, 2022 1:09 am

Do have a bunch of fw filter rules that rely on conn tracking. Traffic is not natted.
thank that could still be the problem?
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2546
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Traffic drops with BGP routing

Sat Oct 29, 2022 1:22 am

make sure you dont have rp-filter enabled

check it on

ip -> settings

also try disabling any firewall rule in chain forward dropping invalid packets
 
DarkNate
Member
Member
Posts: 492
Joined: Fri Jun 26, 2020 4:37 pm

Re: Traffic drops with BGP routing

Sat Oct 29, 2022 2:09 am

rp-filter should be set to loose mode for assymetric routing.

But overall, assymetric routing is bad traffic engineering. What you should do is announce the largest possible aggregates of all your prefixes to both upstreams, equally without prepending or more specifics.

More specifics should be used for DDoS blackholing/scrubbing instead during DDoS mitigation/detection.

Then use something like PathVector for traffic engineering in an automated fashion.
 
User avatar
Cha0s
Forum Guru
Forum Guru
Posts: 1080
Joined: Tue Oct 11, 2005 4:53 pm

Re: Traffic drops with BGP routing

Sat Oct 29, 2022 1:37 pm

Do have a bunch of fw filter rules that rely on conn tracking. Traffic is not natted.
thank that could still be the problem?
Absolutely.
NAT is not relevant to the problem. Doing connection tracking (and thus accepting established/related connections and somewhere down the road dropping the remaining traffic) is.

When the traffic returns on another router, that router knows nothing about that connection, so it is not established/related by its standpoint, thus it gets dropped.
 
texmeshtexas
Member Candidate
Member Candidate
Topic Author
Posts: 118
Joined: Sat Oct 11, 2008 11:17 pm

Re: Traffic drops with BGP routing

Sun Oct 30, 2022 12:33 am

RP was set to loose and I have default route installed from upstream BGP peer so really does nothing. Turned off for now.
 
texmeshtexas
Member Candidate
Member Candidate
Topic Author
Posts: 118
Joined: Sat Oct 11, 2008 11:17 pm

Re: Traffic drops with BGP routing

Sun Oct 30, 2022 12:44 am

Do have a bunch of fw filter rules that rely on conn tracking. Traffic is not natted.
thank that could still be the problem?
Absolutely.
NAT is not relevant to the problem. Doing connection tracking (and thus accepting established/related connections and somewhere down the road dropping the remaining traffic) is.

When the traffic returns on another router, that router knows nothing about that connection, so it is not established/related by its standpoint, thus it gets dropped.
yeah, I do accept est/rel traffic but I dont drop other traffic, just check it against rules like the rate of inbound connections being too high, DNS request coming.
I may try turning any drop rule off and see if it helps.
 
texmeshtexas
Member Candidate
Member Candidate
Topic Author
Posts: 118
Joined: Sat Oct 11, 2008 11:17 pm

Re: Traffic drops with BGP routing

Sun Oct 30, 2022 12:53 am

make sure you dont have rp-filter enabled

check it on

ip -> settings

also try disabling any firewall rule in chain forward dropping invalid packets
thanks, not dropping invalid.
 
texmeshtexas
Member Candidate
Member Candidate
Topic Author
Posts: 118
Joined: Sat Oct 11, 2008 11:17 pm

Re: Traffic drops with BGP routing

Sun Oct 30, 2022 1:02 am

rp-filter should be set to loose mode for assymetric routing.

But overall, assymetric routing is bad traffic engineering. What you should do is announce the largest possible aggregates of all your prefixes to both upstreams, equally without prepending or more specifics.

More specifics should be used for DDoS blackholing/scrubbing instead during DDoS mitigation/detection.

Then use something like PathVector for traffic engineering in an automated fashion.
rp-filter is on loose.
Yeah, that is what I'm trying to work towards. announcing all prefixes to both upstreams without prepending. and I announce DDoS blackholing out both sides also.
Not sure how PathVector or similar would fit in.
 
DarkNate
Member
Member
Posts: 492
Joined: Fri Jun 26, 2020 4:37 pm

Re: Traffic drops with BGP routing

Sun Oct 30, 2022 4:24 pm

rp-filter should be set to loose mode for assymetric routing.

But overall, assymetric routing is bad traffic engineering. What you should do is announce the largest possible aggregates of all your prefixes to both upstreams, equally without prepending or more specifics.

More specifics should be used for DDoS blackholing/scrubbing instead during DDoS mitigation/detection.

Then use something like PathVector for traffic engineering in an automated fashion.
rp-filter is on loose.
Yeah, that is what I'm trying to work towards. announcing all prefixes to both upstreams without prepending. and I announce DDoS blackholing out both sides also.
Not sure how PathVector or similar would fit in.
Step 1: Fix the issue you're facing here
Step 2: Get full tables from upstream transits
Step 3: Enable loose mode rp-filter
Step 4: Do not use stateful-ness on edge routers
Step 5: Announce prefixes aggregated, equally, no prepends to all transits, IXPs and also PNIs/Bilaterals.
Step 6: Install PathVector on-prem server and configure it to auto traffic engineer your routing based on latency and other parameters of your choice.

For other best practices, follow the guide for ISPs:
viewtopic.php?t=176358
 
texmeshtexas
Member Candidate
Member Candidate
Topic Author
Posts: 118
Joined: Sat Oct 11, 2008 11:17 pm

Re: Traffic drops with BGP routing

Tue Nov 01, 2022 12:46 am

I found what I think is the primary issue. As I'm using a collapsed config at the edge, my core network and edge and NAT are the same router.
I have some firewall rules to help protect the network from bogons, bad ports, etc.
one of the rules, and I thought this one was gone long ago, was a PSD (ports scan) rule that was killing some traffic. Removed that rule and things are running much better.

I'm working towards separating border from network and NAT. QoE is at data center and BNG is at tower site. NAT is upstream from QoE box and is tough to implement at BNG due to multiple paths that are available to many BNG units.

Who is online

Users browsing this forum: No registered users and 5 guests