Microsoft definitely takes the cake on useless error messages (There is no cake). While setting up an edge server, I started with this (OCS Protocol Stack, Error 14501, Office Communications Server event log):

A significant number of invalid certificates have been provided by remote IP address 24.18.22.8 when attempting to establish an MTLS peer. There have been 10 such failures in the last 1 minutes.
Certificate Names associated with this peer were

The serial number of this certificate is
.
The issuer of this certificate is
The specific failure types and their counts are identified below.
Instance count – Failure Type
10 C3E93D6D

First off, it’s totally awesome that you don’t get this error right away, you have to fail a bunch of times before it shows up in the event log. I hate certificates, so I’ve been assuming it was that. I ran C3E93D6D through ‘lcserror’ which comes with the resource kit.

lcserror C3E93D6D
0xC3E93D6D -> (SIPPROXY_E_CONNECTION_EXTERNAL_INTERNET_ACCESS_DISABLED) (C:\Program Files (x86)\Microsoft Office Communications Server 2007\ResKit\RTCERes.dll)

Remote user client cannot connect because access is disabled.

Yup. More useful. My client says “cannot sign in because the server is temporarily unavailable. If the problem persists, contact your system administrator.” I thought about calling myself, but I’m an asshole and figured it wouldn’t do either of us any good. Another user on Vista x64 said he was getting a certificate error but I didn’t see it.

OMG! As an aside, if you’re doing this on x64, and you’re doing an edge server in a box not in the domain as I think the whole point is, you have to manage this through the Computer Management mmc snap-in. Long time windows fixers would be tempted to right click ‘My Computer’ and choose manage. Don’t. It doesn’t show up there. Run, or create a shortcut for ‘%windir%\system32\compmgmt.msc -32’. I don’t know how I found that out, deep in a document somewhere.

In here, I went to the ‘External User Access’ section of the status tab and saw “‘Remote User Access:’ was the big off red X. Right click, Properties on the server, Access Methods tab, Check ‘Allow remote user access to your network’. I guess. I mean, is this needed? Why else would I have an edge server? I guess for federation.

I feel like doing this caused a little more ssl traffic to pass, but it’s mostly binary data that means nothing to me. I still see the servers public ssl web cert being passed (via wireshark, remember to turn off chimney) though. A coworker said something about having to disable MTLS on the last version of LCS, but I couldn’t find it. It sort of made sense as the first error seems to be saying that the client is passing a bunk cert. Page 39 of the Edge Server Deployment Guide has a table for firewall configuration that lists:

Local Port: 5061 TCP (SIP/MTLS)
Local Port: 443 TCP (SIP/TLS)

I was thinking at first 443 was all the rage. But after looking in the above deployment guide at a diagram I started wiresharking for traffic on tcp pot 5061 as well and there’s a bunch of TLS crap going on there. Poking around on the ‘internal interface’ I see traffic is actually going to the ‘front end’ server. Running wireshark there, I see some similiar business and more fun errors, getting slightly more useful:

A significant number of invalid certificates have been provided by remote IP address ipaddressofedgeserver when attempting to establish an MTLS peer. There have been 10 such failures in the last 30 minutes.
Certificate Names associated with this peer were
actualinternalservername

The serial number of this certificate is
4e7f38cd00000000001e.
The issuer of this certificate is corp.widemile.com
The specific failure types and their counts are identified below.
Instance count – Failure Type
10 C3E93D6A

Run it through the damn tool again. Couldn’t this just be put in the event log?

lcserror C3E93D6A
0xC3E93D6A -> (SIPPROXY_E_ROUTING_UNKNOWN_SERVER) (C:\Program Files (x86)\Microsoft Office Communications Server 2007\ResKit\RTCERes.dll)
The FQDN in the peer’s certificate subject name is not a configured server.

I recently added the the edge server’s ip on the Host Authorization tab of the Front End Properties for the group (Under ‘Standard Edition Servers’ under the forst in the mmc snap in on the full server, not the edge server). Doing so added the name of the server under ‘access edge servers’ in ‘edge servers settings’ on the status tab for the forest in said mmc snap in. The dns didn’t match though. My dns is mess. The actual place where this comes from though is ‘Global Properties’ on the forest, ‘Edge Servers’ tab.

Followed my more failing, which you gotta keep repeating to get the darn error messages to show up the event log:

The server configuration validation mechanism detected some serious problems.

1 errors and 0 warnings were detected.

ERRORS:
Two server roles at FQDN [myedgeserversfqdn.test.org] have different ‘Throttle As Server’ options. First server has GUID {D00AC7FC-3E15-4CD6-A33A-0A823BE24EA1} and role ‘Authorized Host’ (option is not set). Second server has GUID {C8F1ADE5-9CB5-56AE-BDE3-D53B9D9B3BF9} and role ‘Edge Server’ (option is set).

WARNINGS:
No warnings

Cause: The configuration is invalid and the server might not behave as expected.
Resolution:
Review and correct the errors listed above, then restart the service. You also wish to review any warnings present.

So I go into the server group, right click, properties, front end properties, host authorization tab and set ‘throttle as server’.

This time I get a log in prompt on the click (Office Communicator 2007). Enter credentials, wait a while, and it works!

Older notes:

1) I had a bunch of problems getting past the certificate wizard because I thought importing certificates was enough, but this doesn’t actually assign them to the interfaces.

2) Speaking of interfaces, you need two. Well, two IPs. You can just add an IP Alias to the existing interface (TCP/IP Properties, advanced). Otherwise an early wizard fails with a useless error when it gets to the external interface page. See this forum thread for a little more info.