Hello,
On Fri, 5 May 2000, Andrey Savochkin wrote:
> > We want:
> >
> > 1. only one to reply to the ARP requests for VIP, i.e. the
> > traffic to come to the Director and to be forwarded to one
> > of the real (internal) servers. If all hosts reply to the
> > Router's ARP broadcast requests we have a problem.
> >
> > Filters? OK, the above can be handled using filters. Just
> > stop the ARP replies from the real servers.
>
> You may solve the issue in a more simple way: fix VIP->MAC translation to
> Director's address on your router, so that ARP isn't used at all.
> The filters proposed by Andi work here, too.
Agree, arpfilter stops the replies. But sometimes we
can't set a static ARP entry in the router or may be
sometimes the role of the Director is changed and another
real server becomes a Director. We have to change the ARP
entry again. Anyway, the point 1 can be solved using static
ARP entry in the Router (hardest way) or by setting
arpfilter in the real servers (which is not enough for LVS).
>
> >
> > 2. the real servers must not broadcast ARP requests with
> > saddr=VIP because only the Director must announce VIP. For
> > this the real servers must announce another IP as source
> > address for the ARP request. I.e. requests such as "who-has
> > ROUTER tell VIP" are allowed for the Director only. The
> > real hosts must announce another visible (and uniq) IP
> > address to the router or the reply will not be received.
> > This can't be handled using a filter. This is a logic.
>
> You may easily implement it without any kernel modifications.
> Check how ARP engine selects source address. In the current kernels it picks
> up the address from the same network without "secondary" flag.
LVS doesn't use autoselection. The change in the
autoselection is included only to cover the "hidden"
functionality. The VIP is local address (we can talk IP) but
it is not advertised. I think, the feature is very clear.
Look in arp_solicit. We can set per-device flag if the src
IP address in the header can be announced. This is the
trick which is not present in the arpfilter functionality
(Andi, I see, the hidden functionality is not replaced from
arpfilter) nor in fib_local_source (route.generic). I.e. the
feature to hide some local IP addresses by not including
them in the ARP requests, addresses which are shared in the
LAN. This feature covers such shared IP addresses. It can't
be tuned at routing level, at least this will be more
difficult: to disable some local addresses to be announced
in the ARP requests.
>
> Select which address do you want to see by setting up this address as the
> primary. If you may choose IP address for Director->node communications from
> the different network than node->router communications, then it's done.
>
The packet from the Director to the real server is
not altered. It is just forwarded. The IP header is not
changed.
> Otherwise, you may set IP address for node->router communications as primary
> and address for Director->node communications as secondary, and assign proper
> preffered sources for all routes. Not very hard, too.
LVS doesn't use the autoselection at all. The first
my patch was named "arp_invisible" and these two points were
handled. Later the feature included the change in the
autoselection and allowed replies to the unicast requests.
So, IFA_F_SECONDARY flag can't help here (for point 2).
The hidden flag is used only in the real servers. In
the Director this flag is not set. The VIP is advertised
only in the Director and there are no changes in the
Director.
>
> BTW, I fix the plain-and-dumb ARP source selection in my patch.
> ftp://ftp.sw.com.sg/pub/Linux/people/saw/kernel/v2.3/route.generic
> (look at arp.c changes).
I see. May be the fib_local_source call will be
replaced with arp_local_src? May be we have problem here? We
need a way to skip some local addresses and not to announce
them as source of the ARP request: the shared local
addresses.
> When I manage to integrate it into the mainstream kernel, this problem will
> be solvable even more simplier.
ARP replies can be filtered. If we can found a
solution for point 2, that will be good. Now the solution is
under the "hidden" device semantic. I will be happy if we
can find another semantic, i.e. not to announce specific
local IP addresses in the ARP requests. LVS doesn't need
help from the autoselection code.
>
> In any case, you don't need special kernel hacks to do implement your
> requirement #2.
How can we select which local IP addresses can't be
announced? The saddr is already fixed in the outgoing
packet. We have to construct ARP request with non-shared
(uniq) source address. We can select any address from the
outgoing device. Here is the point. We don't care about the
"secondary" flag for the ougoing device (where the ARP
request is sent). This device is not hidden. Even if it is
hidden we don't stop replies to the same device as the
device used for the requests. In fact, we hide one device
from the other devices, for example, the IP addresses on
dummy0 from the ARP requests coming through eth0. Using
eth0/arpfilter we can ignore the requests from the Router
coming from eth0 and asking for any local IP addresses not
on eth0. This is in the real server. But we can't stop
announcing addresses configured on dummy0 in the ARP
requests "who-has ROUTER tell VIP" which exit from eth0
(the replies come again from eth0). In arp_solicit we treat
these addresses as "remote"! This is something like the
transparent proxy. The rule is: we can't announce addresses
which are not local or addresses which are shared on the
LAN.
Andi, is my interpretations for the arpfilter
correct?
The complete solution: use static ARP entries on all
hosts :) But that is not always possible or it is very hard
to maintain.
>
> >
> > These are the minimum requirements from this
> > functionality for the LVS project. The LVS users hide "lo",
> > "dummy*" and even "tunl*". In fact, we want to hide IP
> > addresses, not devices.
>
> That's my point!
> The solution in 2.2 kernels tries to solve problems messing about wrong
> objects, sorry for being frank.
May be. But I don't understand where is the problem.
We have a per-device flag which is off by default.
> If you do not like existing features, add special address flags.
> Or add special table for ARP engine specifying what address to use and which
> replies to ignore. Let's do it in a natural way :-)
With arpfilter + hidden we have all variants to play
with ARP: to make a device to mute about the other devices
(arpfilter) and to mark a device as hidden from the all
other devices (hidden).
Only with arpfilter (without hidden feature) the
real servers will continue to announce shared IP addresses
in the ARP requests. Is there a way to solve this problem?
I think arpfilter + hidden are the best thing we can have.
Regards
-- Julian Anastasov <uli@linux.tu-varna.acad.bg>- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Sun May 07 2000 - 21:00:17 EST