Jump to content

Network

From NYC Resistor Wiki

Overview

[edit | edit source]
Network rack with an old switch in the middle of two patch panels

The network consists of a cable modem feeding an OpenWrt router/AP, which in turn uplinks to the mezzanine Ubiquiti managed switch. From there, connections run to two loft patch panels: one Blue, one Grey. Each network drop in the space has a corresponding Blue and Grey jack; only some are wired through to the switch, and only some are currently used. The main table has a small managed switch.

Blue and Grey cables are functionally identical. The colours reflect physical routing rather than logical separation.

Static Devices

[edit | edit source]

Most fixed equipment uses static DHCP leases on the OpenWrt router. If you add a static IP address, give it a static DHCP lease on the openwrt router, even if the device doesn't use dhcp. These are the main devices:

Device IP Address
OpenWrt loft router 192.168.0.1
Home Assistant 192.168.0.2
OpenWrt table AP 192.168.0.3
Lazzor 192.168.0.4
OpenWrt lobby AP 192.168.0.6

Patch Panel Map

[edit | edit source]
Cabling map: Grey (left, top) ↔ Blue (right, bottom)
Grey device / location Switch port Label Switch port Blue device / location
Lobby black cable (unterminated) 1 1 Laser Loft 2 Lobby AP
Lazzor 3 2 Laser Room 4 Laser room computer
(unused) 5 3 Craft Shelf Arch 6 (unused)
(unused) 7 4 Craft Shelf Top 8 Payphone
(unused) 9 5 Cubesville 10 Denon AVR
Game of Life cube 11 6 Main Room Beam Front 12 Main table Ethernet (phone, switch)
(unused) 13 7 Main Room Beam Back 14 Main Room AP
(unused) 15 8 Tool Town L (breaker panel) 16 Sherline computer
(unused) 9 Tool Town R (shopbot) (unused)
(unused) 10 Mezzanine R (unterminated) (unused)
(unused) 11 Mezzanine L (unterminated) (unused)
(unused) 12 Main Room Couches (unterminated) (unused)

Unconnected Grey ↔ Blue runs: 13–24

Router

[edit | edit source]
Port Connected device
WAN Spectrum modem
LAN 1 (Woody's undocumented HAM stuff?)
LAN 2 Mezzanine switch (uplink)

Mezzanine Switch

[edit | edit source]

The ports on the left are for the patch panel.

The SFP ports on the right are:

17 Home Assistant (purple) 18 Uplink (yellow)

The switch is managed through the Ubiquiti app in Home Assistant

ToDo

[edit | edit source]
  • Ubiquiti setup

OpenWRT router and Wi-Fi access points

[edit | edit source]

The space has three Comfast CF-WA933 running OpenWRT: one router and two wired access points. All three provide Wi-Fi. The main router supplies Internet access, DHCP, DNS, IPv6 and the NYC Mesh connection over vpn; the other two extend coverage.

The two APs are powered by PoE; Main is powered by a 12V barrel jack adapter.

Device Where it is
Main router Loft
Satellite Above the hack table
Lobby Above the front door in the lobby

Hostnames and IP addresses are in the shared password vault.

Administration is by SSH as root or LuCI. The AP addresses are reservations on main, so changing or replacing an AP's DHCP identity also requires updating main.

Two networks, two exits

[edit | edit source]
NYCR / NYCR24 / NYCR-IoT
    -> LAN: VLAN 1, 192.168.0.0/24
    -> main (192.168.0.1) -> cable modem -> Internet

-NYC Mesh Community WiFi-
    -> public mesh: VLAN 30, 192.168.30.0/24
    -> main (192.168.30.1) -> wgmesh (WireGuard) -> NYC Mesh SN3

The private SSIDs all join the same LAN. NYCR-IoT is not an isolated IoT network. The public mesh SSID is on a separate VLAN and firewall zone, with client isolation enabled on its BSSes.

The main router provides DHCP/DNS for both networks. It routes LAN traffic to cable WAN and public-mesh traffic through WireGuard.

Mesh depends on three pieces agreeing: meshvpn/wgmesh firewall policy, the PBR policy "NYC Mesh SSID through WireGuard", and the tunnel/routes in network config. There is also an explicit 10.0.0.0/8 route through wgmesh. WireGuard does not automatically install its peer's allowed-IP routes here.

Main's Internet interfaces are wan and wan6; its internal ports are lan1 and lan2. The modem is reachable (only ping) at 192.168.100.1 through main's cablemodem interface (192.168.100.2/24).

Between the router, switches and APs, VLAN 1 is native/untagged and VLAN 30 is tagged. Each AP bridges all three wired ports. An AP's port labelled WAN is part of this internal bridge. The APs do not route VLAN 30 or run DHCP/DNS servers.

IPv6 access policy is explicit in main's firewall: there are inbound LAN-forwarding and router-management exceptions. Include those rules when reviewing external access; the IPv4 NAT policy is not the whole access policy.

Wi-Fi

[edit | edit source]
SSID Bands Authentication Network Purpose
NYCR 2.4 and 5 GHz WPA3-Personal / SAE LAN 'Main' network. What you should be using.
NYCR24 2.4 and 5 GHz WPA2-Personal LAN WPA2 compatibility; no 802.11r fast transition. Legacy network, both bands despite the name
NYCR-IoT 2.4 GHz WPA2-Personal LAN For IoT devices inside Resistor. Backwards compatibility over speed.
-NYC Mesh Community WiFi- 2.4 and 5 GHz Open Public mesh / VLAN 30 NYC Mesh; can't access the Resistor internal network

NYCR has fast roaming configured across all three devices. The two NYCR BSSes per device have unique MAC/NAS identities; their peer tables and shared roaming keys must be maintained together across all three devices.

usteer handles inter-AP information/steering; it is separate from the fast-roaming peer configuration.

Configured radio plan Main Satellite Lobby
2.4 GHz 6 / HE20 1 / HE20 11 / HE20
5 GHz 36 / HE160 100 / HE160 149 / HE160

After a radio change, verify the operating channel and width rather than just the saved setting.

Where changes belong

[edit | edit source]

Most settings are under /etc/config/:

Change Device(s) Files / dependencies
WAN, modem, VLANs, IPv6 or WireGuard Main; VLAN bridging also on APs network
DHCP reservations, local DNS or upstream resolvers Main dhcp
Mesh egress or network access policy Main network, pbr, firewall
SSIDs, passwords or roaming All three wireless; usteer for steering policy. Preserve per-BSS identities.
Home Assistant monitoring access Affected device(s) rpcd, luci and their integration account/ACLs
Hostname or location notes Affected device system

If something breaks

[edit | edit source]
Failure boundary Look here
All APs affected, including wired LAN clients Main is the shared dependency: WAN/modem for Internet, DHCP/DNS for client configuration. Keep AP configuration out of the initial investigation.
New clients can't get an IP address Check the DHCP service on Main
Clients can't resolve hostnames Main's dnsmasq and upstream resolvers; distinguish local-name failures from external DNS failures.
Public mesh fails but private LAN works Separate VLAN 30 transport from main's mesh DHCP, WireGuard, PBR and firewall. A tunnel handshake does not prove the SSID's egress route.
Public mesh fails on only one AP Its SSID-to-VLAN attachment and the tagged VLAN 30 path through the Ubiquiti switches.
Association works but moving between APs fails The affected BSS identities and roaming peers/keys; then effective steering behaviour.
Only HTTPS management fails That device's certificate paths/renewal and DNS/challenge reachability; SSH is the alternative access path.

Adding a device

[edit | edit source]

To add another access point:

  1. Give it a unique hostname and management identity. Add its DHCP reservation on main; leave the AP's management interface as a DHCP client. Assign a unique IPv6 identity/suffix as well.
  2. Configure its bridge and upstream switch port to carry native/untagged VLAN 1 and tagged VLAN 30. Keep DHCP/DNS servers disabled on the AP; it bridges both networks rather than providing another routed WAN.
  3. Use an existing AP's shared SSID/security settings, with credentials from the shared vault. Give every BSS a unique MAC address and choose its radio channels as part of the space's channel plan. Do not duplicate the source AP's management or BSS identities.
  4. Give its two NYCR BSSes unique NAS IDs, following nycr-LOCATION and nycr-LOCATION-2g, and keep mobility domain 1337. Add both new peers to the r0kh/r1kh tables on every existing device; the new AP needs the complete peer set too. Preserve the shared roaming keys and reload Wi-Fi on affected devices after coordinating the updates.
  5. Use a full wpad variant and enable usteer on LAN, matching the other APs.
  6. Verify management access after reboot, private-LAN DHCP/DNS and Internet access, public-mesh DHCP and WireGuard egress, and roaming to/from the existing APs. Check both radio bands.
  7. Record its management details in the vault, add its location and channels to this page, record the switch port and PoE source, and take a backup.

Replacing a device

[edit | edit source]

Restore a backup for that logical device. An AP replacement must retain both VLANs, DHCP/DNS disabled, integration access and the intended BSS/roaming identity. Update main's DHCP reservation if the replacement's management identity changes. Do not connect old and replacement units with duplicate BSS identities.

Main's replacement also needs all DHCP reservations, WAN/modem and IPv6 configuration, WireGuard credentials, PBR and firewall policy. Verify private LAN Internet access and public-mesh egress independently. Keep the recovery image and procedure with the device's private backup.


Dynamic DNS

[edit | edit source]

NYC Resistor has dynamic IPv6 DNS for selected hosts under: 

*.dyn.nycresistor.com

The public IPv6 prefix assigned by Spectrum changes every few weeks. Hosts on the Resistor LAN have stable IPv6 host IDs, so when the /64 changes we update the network prefix of all published AAAA records while preserving the lower 64 bits.

The low 64 bits of each AAAA record are persistent. Most of these correspond to hostid entries configured in OpenWrt.

The initial AAAA record in deSEC must therefore contain the correct host ID. Future updates only replace the /64 prefix.

For example: 

espressomachine.dyn.nycresistor.com
    2603:7000:1234:5678::cafe

The hostid is cafe

If the ISP changes our prefix from:

2603:7000:1234:5678::/64

to:

2603:7000:90ab:cdef::/64

the DNS record becomes:

2603:7000:90ab:cdef::cafe

Adding a host

[edit | edit source]
  1. Give the device a stable IPv6 host ID in OpenWrt.
  2. Create its AAAA record in the dyn.nycresistor.com deSEC zone with the correct host ID.
  3. Add its FQDN to /config/ddns-hosts.txt.
  4. Run the Resistor DDNS automation manually from Home Assistant.
  5. Check that the resulting AAAA record contains the current Resistor /64 and the expected host ID.

DNS provider

[edit | edit source]

dyn.nycresistor.com is delegated separately from the main nycresistor.com DNS zone and is hosted by deSEC. This means the dynamic-DNS credentials cannot modify unrelated Resistor DNS records.

The credentials are in the usual place.

Home Assistant general setup

[edit | edit source]

Home Assistant updates the prefix at deSEC through the "Resistor DDNS" automation. It runs on boot, on prefix change and weekly.


Hosts which should be published are listed one per line in /config/ddns-hosts.txt. Blank lines and lines beginning with # are ignored. Changing this file does not require restarting or reloading Home Assistant.

Do not add arbitrary LAN devices here just because they have a stable IPv6 address. This list is for things which should actually have a public DNS name.

Update process detail

[edit | edit source]

Home Assistant has sensor.resistor_ipv6_prefix, a command-line sensor which looks directly at interface end0: 

ip -6 -o a s dev end0|awk '$4~/^[23]/{split($4,a,":");print a[1]":"a[2]":"a[3]":"a[4]"::/64";exit}'

Home Assistant then has a shell_command.update_resistor_ddns command.

It:

  1. reads /config/ddns-hosts.txt;
  2. removes comments and blank lines and joins the names with commas;
  3. sends one batched request to deSEC;
  4. passes sensor.resistor_ipv6_prefix as the new IPv6 /64;
  5. tells deSEC to preserve IPv4 records (currently none).

The deSEC API token is stored in Home Assistant's secrets.yaml, not in an automation.

The command uses roughly: 

hosts="$(grep -Ev "^[[:space:]]*(#|$)" /config/ddns-hosts.txt | paste -sd, -)"

curl -fsS \
  -H "Authorization: Token verySecret" \
  --get \
  --data-urlencode "hostname=$hosts" \
  --data-urlencode "myipv4=preserve" \
  --data-urlencode "myipv6=$prefix" \
  https://update.dedyn.io/

Batching all hosts like this matters: an earlier setup updated records individually and ran into deSEC rate limiting.

The Resistor DDNS Home Assistant automation runs when:

  • sensor.resistor_ipv6_prefix changes;
  • Home Assistant starts, after a short delay;
  • once a week as a sanity/reconciliation update.

When Spectrum gives us a new IPv6 prefix switch it causes: 

OpenWrt receives new IPv6 prefix
        ↓
HAOS receives address in new /64
        ↓
sensor.resistor_ipv6_prefix changes
        ↓
Resistor DDNS automation runs
        ↓
one batched deSEC request
        ↓
all published AAAA records get the new /64

The periodic update is only a fallback in case a prefix-change event was missed.

ESP32 near elevator door

[edit | edit source]

(TODO new page dedicated to space access controls)

Currently, there's a m5stack ATOM-Lite (very small) ESP32 board hanging above the elevator door/intercom/laser loft stairs area. This serves:

  • Elevator door sensor inside(?) maglock housing (it is a reed switch)
  • Keypad in elevator shaftway (a direct wired 4x3 key matrix on the end of a cat5 cable)

ATOM-Lite publishes these into HA, where the sensor states are used in various automations related to locks. I think it publishes the elevator door sensor state over Bluetooth (why?) and publishes codes entered into the keypad over MQTT. See the config in ESPHome though.

Wiring (rat nest) "diagram":

[Pictures are worth 1000 words, but only if you can upload them. --Ed]

jumpers:

  • ATOM-Lite GND <=> WAGO <=> white twisted pair a terminal <=> elevator door sensor (reed switch)
  • ATOM-Lite GPIO21 <=> not connected
  • ATOM-Lite GPIO 19 <=> grey jacket cat5, green-white <=> keypad terminal 5 (col 3) [2]
  • ATOM-Lite GPIO 23 <=> grey jacket cat5, green <=> keypad terminal 6 (col 2)
  • ATOM-Lite GPIO 33 <=> grey jacket cat5, brown-white <=> keypad terminal 7 (col 1)

breadboard:

  • ATOM-Lite 3v3 <=> 1 kΩ pull-ups to each of GPIO 25,32,26,33 [1]
  • ATOM-Lite GPIO 21 <=> pull-up & grey jacket cat5, blue-white <=> keypad terminal 1 (row 1) [2]
  • ATOM-Lite GPIO 26 <=> pull-up & grey jacket cat5, blue <=> keypad terminal 2 (row 2)
  • ATOM-Lite GPIO 32 <=> pull-up & grey jacket cat5, orange-white <=> keypad terminal 3 (row 3)
  • ATOM-Lite GPIO 22 <=> pull-up & grey jacket cat5, orange <=> keypad terminal 4 (row 4)

[1] Without these strong external pull-ups, the ~2meter twisted pairs run to the keypad is too noisy for the matrix scanning to work. Ask me how I know. 3.3 kΩ is better than the 45 kΩ (?) on-chip pull-ups but still not enough.

[2] here, 4 rows and 3 cols of the physical keypad layout; the ESPHome config calls the physical columns "rows" (pins configured as strobed outputs) and calls the physical rows "columns" (pins configured as inputs)

"Shelly" near elevator door

[edit | edit source]

Switches 12v power to the elevator door maglock...

Retrieved from "https://wiki.nycresistor.com/w/index.php?title=Network&oldid=11193"