SXAccessSXAccessDocs
Open dashboard

Trace why a connection is blocked

The agent can push an invented packet through the same firewall stages a real one meets and print what each stage decided. Run it on the receiving device when two devices see each other but a connection between them still fails.

sxaccess info -d answers whether two devices see each other. It does not answer why a connection between them is refused, because that decision is made by the agent on the receiving device, packet by packet, and it leaves nothing behind to read afterwards.

sxaccess diagnose trace closes that gap. You describe a packet, the agent runs it through the stages a real packet goes through, and each stage prints what it decided. Nothing is sent on the wire: the packet exists only inside the agent, so tracing a port that nothing is listening on is safe and tells you about the rules rather than about the service.

Access rules are checked where a packet arrives, so run the trace on the device you are trying to reach, as an in trace from the other device's address. An out trace never consults the access rules: it ends with Outbound packet, allowed whatever they say.

Find where the packet is dropped

  1. 1

    Go to the device that refuses the connection

    Open a shell on the device you are trying to reach - the database server, not the laptop that cannot get to it - and check that its agent is connected. A trace needs a connected agent. It does not work on a Linux device whose sxaccess info -d shows Kernel on the Interface row, nor on a device started with --disable-firewall; see Limits.

    shell
    sxaccess info
  2. 2

    Find the source address

    A trace is written in overlay addresses, not names. The destination is this device, so you can write self. The source is the address of the device that cannot connect: run the command below on that device, and it prints just the address. If this device lists the other one, you can stay here instead: sxaccess info -d --filter-by-names laptop shows it on the Address: line.

    shell
    sxaccess info --ipv4
  3. 3

    Replay the opening packet

    Describe the packet the other device would send. in is a packet arriving here, followed by the source address and self. --dport is the port of the service, and --syn marks the opening packet of a TCP connection rather than part of one already running.

    shell
    sxaccess diagnose trace in 100.72.0.31 self -p tcp --dport 5432 --syn
  4. 4

    Read the last line first

    Result: is the answer, ALLOWED or DENIED. Above it are the stages the packet passed, in order, each with what it decided. The last stage before Completed is the one that settled it, and it tells you what to change.

Arguments and flags

The command is sxaccess diagnose trace <in|out> <from> <to>. The three arguments are required and come in this order; everything after them is a flag.

Argument or flagWhat it means
in, outin for a packet arriving at this device, out for one leaving it. Only in is checked against access rules. Any other word is rejected
<from>The IPv4 address the packet comes from. self means this device
<to>The IPv4 address it is addressed to. self means this device
-p, --protocoltcp, udp or icmp. Defaults to tcp. icmp is what ping uses; ports do not apply to it
--sport, --dportSource and destination port, for TCP and UDP. Each one you leave out becomes a random port from 49152 to 65534
--syn --ack --fin --rst --psh --urgTCP flags to set on the packet

Reading the output

The first line repeats the packet back to you as you typed it, so you can check you described the one you meant. Then comes one line per stage, then the verdict. This is a connection to a database that no access rule allows, traced on the database server:

shell - a denied connection
$ sxaccess diagnose trace in 100.72.0.31 self -p tcp --dport 5432 --synTracing TCP 100.72.0.31:52134 to self:5432Received: Got TCP packet 100.72.0.31:52134 -> 100.72.0.7:5432Connection Tracking: Not part of a tracked flowRouting: Addressed to this hostDevice ACL: Denied by device ACL (<no id>)Completed: Dropped: denied by ACLResult: DENIED

The stages

A packet does not visit all of them. One addressed to this device meets Routing and then Device ACL; one for a network behind this device meets Routing, Route ACL and Forwarding. Which stages appear tells you how far it got, and that is usually enough to name the cause without reading a log.

StageWhat it is deciding
ReceivedThe packet as the agent read it, with self replaced by this device's address. If your addresses or ports are wrong, you see it here
Connection TrackingWhether the packet belongs to a connection this device opened. If it does, it is allowed here and the trace ends. This is why replies need no access rule of their own
RoutingWhether the packet is addressed to this device or would be routed on to a network behind it. Routing is off means this device routes for no one
Device ACLFor a packet addressed to this device: whether an access rule lets the source reach it with this protocol and port. In brackets is the ID of the access rule that matched, or <no id> when none did
Route ACLFor a routed packet: whether an access rule lets the source reach that address. Where the operating system does the routing, the stage says The kernel routes this packet, route ACLs are not checked here
ForwardingWhere an allowed routed packet would be handed on
CompletedThe verdict and its reason, for example Dropped: denied by ACL. An out trace goes straight here and normally reads Outbound packet, allowed

When the verdict is DENIED

Stopped at Device ACL with <no id>: no access rule lets the source reach this device with that protocol and port. Nothing gets through until a rule allows it, so check the groups both devices are in and the rule's protocol and ports. If a rule should match, check its device requirements: a source device that fails them is left out of the rule, and the trace then looks exactly as if there were no rule.

Stopped at Routing with Routing is off: the address is not this device's own, and this device routes for no one. Run the trace on the routing device of that network instead. Stopped at Route ACL: this device does route, but no rule lets the source reach that address through it; see Networks and resources and Route private networks.

If the trace says ALLOWED and the connection still fails, the rules are not your problem. Check that something is listening on that port and that the device's own firewall lets the connection in, then move on to troubleshooting the client.

Limits

  • A connected agent. Without one, the command fails with the agent is not connected yet.
  • IPv4 only. Both addresses must be IPv4 addresses or self.
  • Not where the system firewall enforces the rules. On a Linux device whose sxaccess info -d shows Kernel on the Interface row, the operating system's firewall applies the access rules, not the agent. The trace then fails with this device's firewall layer cannot trace packets. Only the detail view prints that row.
  • Not without the packet filter. On a device where sxaccess start ran with --disable-firewall, the agent loads no packet filter, so there is nothing to trace. The command fails with the agent has no firewall layer running.
  • An out trace changes state. It passes through the real outbound filter, so it records a connection just as a real packet would. A later in trace of the reply then matches that connection and is allowed at Connection Tracking.

Related