Hi, folks! I’ve learned a lot these past 8-9 months in self-hosting, but I’m not sure what’s going on with my reverse proxy setup. At this point, I’m only running Nginx Proxy Manager, Jellyfin, and Komga, separated from my home network via VLANs and firewall rules. My home network is 192.168.1.X and hosted services are on 192.168.10.X. Yet when I look at access logs for Jellyfin or Komga, both are reporting access from 192.168.65.1 when accessed from outside my home network. I don’t have a 65 subnet, so this is strange. The reverse proxy is run on a mini PC at .10.201, and Jellyfin and Komga are hosted on a NAS at .10.202; all of them run via Docker containers.

The instructions for Jellyfin for reverse proxy just say to list the address for the reverse proxy, which I did using IPv4 and the domain, and various configurations of only one or the other, as well as the 192.168.65.1 address that manifested out of nowhere. I haven’t observed any kind of change in IP address reporting after making changes here. I haven’t begun to try to configure Komga for the reverse proxy, because I figure until I get it working for Jellyfin and understand how to do it, it probably doesn’t matter; I only listed it here as evidence that two different services were reporting the same phantom IP address. I did ask for help on this issue on a Discord server a little while ago, and some of those folks suggested looking at the Docker networks. I left network settings on basically default for all three of these applications, and they all seem to generate something like a 172.17.0.X address, not the 192.168.65.1 that I’m seeing in logs.

I know there are supposed to be extra headers on network packets that these services can be aware of if they know it’s going through a reverse proxy first, and I figure this is important for proper monitoring of folks accessing my hosted services, so i want to get this figured out before I start running a few more apps. Any help anyone can offer would be appreciated. A lot of times, the documentation for this stuff is written up with an assumption of more understanding than the person reading it might have. Or maybe I just glanced right by something obvious and stupid.

  • treadful@lemmy.zipEnglish
    8·
    5 days ago

    You’re gonna want to configure nginx to send along an X-Forwarded-For HTTP header (there’s also Forwarded, but it’s not as well supported yet). An example in a location block:

    location / {
    proxy_pass http://backend/;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

    Looks like Jellyfin supports it.

    • ampersandrew@lemmy.worldOPEnglish
      1·
      5 days ago

      I’m not quite sure where this goes. It’s Nginx Proxy Manager (in case that affects your answer compared to vanilla nginx), and I’m using Docker Desktop for much the same reason: I’m doing as much configuration via GUI as possible. Is this a Docker configuration or an Nginx Proxy Manager configuration? There is a “Custom Nginx Configuration” screen for my Proxy Host; does it go there? In a nested location block?

      • treadful@lemmy.zipEnglish
        2·
        5 days ago

        I really don’t know shit about the GUI tools, but presumably they overlay on the same configuration.

        The config I posted would be part of the nginx configuration. You’re instructing it to append the client IP to the X-Forwarded-For header.

        There is a “Custom Nginx Configuration” screen for my Proxy Host; does it go there? In a nested location block?

        I doubt it, but I’m not sure. The location block would normally be where your reverse proxy configuration would already be, and you’d need to just add that one line. But again, I don’t know how that translates to these GUI tools. Maybe if you search for proxy_add_x_forwarded_for and “Nginx Proxy Manager” you might find what you need.

        • ampersandrew@lemmy.worldOPEnglish
          1·
          5 days ago

          I know you said you don’t know anything about the GUI equivalent, so I’m mostly responding to this comment to publicly post how far I’ve gotten off of your tips. There is a “Custom Locations” tab for my proxy host in NPM, but even after attempting to follow this guide I found, I can’t get my custom locations to persist after hitting the Save button.

          EDIT: I was leaving out the path on the location, so now that that’s corrected, it saves my changes, but I’m not seeing any progress in the IP address reporting.

          • badlotus@discuss.onlineEnglish
            1·
            4 days ago

            Try setting this in advanced settings:

            proxy\_set\_header Host $host;

proxy\_set\_header X-Real-IP $remote\_addr;

proxy\_set\_header X-Forwarded-For $proxy\_add\_x\_forwarded\_for;

proxy\_set\_header X-Forwarded-Proto $scheme;

            • ampersandrew@lemmy.worldOPEnglish
              1·
              4 days ago

              Still the same problem, I’m afraid. I also experimented with putting some of this stuff in the Custom Locations section of NPM’s proxy host settings, since that’s how it looks in Jellyfin’s nginx guide. The only feedback that I got at any point that my changes in NPM were doing anything at all was when my mobile app failed to connect to my server because of how I’d mucked around in there; I did delete those changes before adding your suggestions, of course.

  • Ooops@feddit.orgEnglish
    5·
    5 days ago

    the 192.168.65.1 address that manifested out of nowhere

    Or it’s indeed the internal virtual lan your docker config uses…

    Is that an actual problem or are you really just trying to fix the logs? That should be doable via headers so the real origin is preserved independent of proxying.

    • ampersandrew@lemmy.worldOPEnglish
      1·
      5 days ago

      Yeah, that appears to be the case from a link someone else linked here, but I was unable to trace that from anywhere I could find in my containers before.

      I am trying to fix the logs so I can see the origin of connections to my services, but I’ve been unsuccessful in getting additional configurations to resolve that.

      • Ooops@feddit.orgEnglish
        2·
        5 days ago

        Well… it’s the default iirc (192.168.65.0/24). So that’s what docker uses if there is no explicit configuration you could find.

        • ampersandrew@lemmy.worldOPEnglish
          1·
          5 days ago

          It appears to be, but I didn’t know that until today. And beyond that, I’ve been trying every permutation of both that IP and the IP of the server on my network in my Jellyfin proxy settings, and it still hasn’t helped to surface the origin IP address.

  • Guiorgy@lemmy.mlEnglish
    1·
    4 days ago

    The way a reverse proxy works is to literally stand in the middle of a connection, so either side only sees them communicating with the proxy, oblivious to the actual endpoint. The only “solution” to this, though it’s more of a hack really, is to somehow send the original IP as additional data in a way that both sides agree on, e.g. the X-Forwarded-For header as others have mentioned, however, as I’ve said, both sides need to support and aggree with that, so you may have to find a setting/config to enable on the services behind the proxy. Think of it this way, if services blindly trusted that header and you weren’t using a reverse proxy, a bad actor could send a request to your service with that header included thereby spoofing their identity in your logs, therefore, this kind of thing is usually opt-in

    • ampersandrew@lemmy.worldOPEnglish
      1·
      5 days ago

      My containers are run in Docker on the metal, not in a VM. My network settings are defaults, which ends up being bridge mode. And yes, my router forwards 443 to NPM listening on 443. It all works except that it’s losing the origin IP address in the logs.

  • Pomal@sh.itjust.worksEnglish
    22·
    5 days ago

    Draw a network map!!! It’s what I’ve done on every security engagement, ever. Mark out all your IP addresses, draw lines to denote the communicating RHP (random high port) against the well-known, e.g. 443/tcp HTTPs, for networks of hosts to servers. Then draw boxes around those IP spaces that are colocated on the same box. Then draw red lines perpendicular to every connection from a client to server that isn’t bounded on a single host.

    The red lines denote non-localhost traffic - places to be observed, and for the most part the only IP:port assignments you should be relying on for routing.

    My rec, draw the map and if that doesn’t help get back to us w it - or feel free to dm me.

    Edit: I’ll find and post an example as a response this thread later.

    • ampersandrew@lemmy.worldOPEnglish
      11·
      5 days ago

      I’ve got a document labeling everything, but I’ve kept my circle small and only expanded out as I conquered the last challenge. The reason I haven’t installed a third service yet is because I don’t have the IPs properly reported yet. I know exactly which machine, IP address, and port each service can be found at.

        • ampersandrew@lemmy.worldOPEnglish
          12·
          5 days ago

          I honestly don’t think I have to. I’ve only got two machines, and I don’t think it’s going to expand any farther than that. And I’m not sure how that helps me with this problem.

            • ampersandrew@lemmy.worldOPEnglish
              1·
              5 days ago

              I’m not trying to be rude. I genuinely don’t see the path between that map and the solution to this problem. Do you need it for your benefit to assist me?

              EDIT: This is the best I can do against your instructions with no example. There’s a fair bit of your instructions I don’t understand. 443 is forwarded from the router to the mini PC in this diagram. I don’t know how to draw red lines perpendicular from the client PC to the mini PC, but that’s where it’s going from and to. I’m more concerned with traffic coming from the internet and making its way to Jellyfin.