atcomsystems.ca/forum Forums VOIP Phones & IP PBX VoIP & Cloud Phones, Hosted PBX, General IP Phones Help! Can't network system anymore!

We have a customer using two Vodavi XTSc systems that have been networked using VoIP for over a year. One is on the North end of town and the other is on the South end. Both sites are connected using cable modem service with the same cable TV provider (Cox).

We relocated the system that was at the South end to a new location right across the street from the original one. The cable people set them up with a new modem, new IP addresses, etc. We reconfigured both switches to accommodate but they won't talk anymore.

We can ping the IP card in both switches from outside just fine. The problem we have found is that both sites are now on the same gateway and subnet. According to Vodavi, this won't work because the systems can't be on the same subnet. They "know" that the IP is in the same subnet so it doesn't try to go out over the Internet. They claim that we must either have Cox place the second location on a different subnet or switch to another ISP at the second site to ensure a different gateway for each site.

Cox won't do this because there is penty of space left in the current subnet and they aren't going to open one up just for us. There's been talk of "bridges" being installed to accommodate this, but again, Cox won't have any part of it.

I don't know who to believe now. None of it makes any sense to me, but then, I am not very savvy when it comes to this stuff. All I know is that when the IP's were 68.XXX.XXX.XXX and 24.XXX.XXX.XXX, everything worked fine. Now everything is 24.XXX.XXX.XXX at both ends with the same subnet.

Can anybody help? The customer is not going to pay for additional hardware, nor are they going to switch to another ISP at one side. We quoted them a firm, fixed price to relocate the system assuming that this would be a piece of cake.

Can't you assign different static IP's to the network cards and have the node pointed to the hub??

This would be the same as all PC'S on the same domain.

And since your two sites are now on the same gateway and subnet there is no need to go outside the infrastructure and hit the IP cloud, thus it is now intranet and not internet.

We do have different static IP's for each network card. It's the fact that these IP's are in the same subnet that is apparently causing the problem.

I'm thinking a cheap cable router/firewall (netgear or linksys) on one side of the connection. That system gets a 192.168.x.x subnet (with a completely different subnet mask too - 255.255.255.0). Then in the firewall, set the DMZ (De-Militarized Zone - basically unrestricted incoming traffic) address to the system's address.

Thanks, Rob. I agree with you on that, but the systems don't currently work in a NAT environment; everything must be full public IP addresses. So far, the cable company can't understand this limitation, nor can I. It's more frustrating to me because I don't know enough about it to have any influence.

What about setting the gateway for each of the systems to be the other systems ip address?. SO for example if the systems are 192.168.1.100 and .101 and you are currently using .1 as the gateway switch it to .100 or .101. If the system doesn't find the other one it should look to the gateway address and find the other system there.

I would have thought Rob was right. Have you tried to put a router in front of one of the systems just to see what it would do Ed?

Both systems are connected directly to the cable modem; there is no router involved so we are relying upon the ISP as the gateway. Like I said, we just moved the existing equipment and didn't add or change much of anything except the IP address for the new site.

I get you. But what Rob is saying is that you can connect a router to the cable modem with the static ip address from the isp, then change the ip address going to the system so that is in another ip scope.

Ed,
Are they using any VoIP phones outside of the building?

Larry

Yes, Larry I am afraid so. The president of the company has one at his home, and of course that has been working fine as well. It still is because it's linked to the original (hub) site.

Bobby:

I don't think that's an option because Vodavi doesn't support NAT. Isn't that what's going to happen if I put one system behind a router? Even with DMZ, any modification isn't supported by this system.

Maybe I am wrong, and I probly am, but we have never been able to do any IP networking with Vodavi systems in over two years without bypassing routers and using small switches ahead of them using static IP's.

We only have one system of dozens where this wasn't necessary, and that customer has a ten-person full time IT staff. For the little customers, it's up to us and we are telephone men, not IT people.

Ed
Are you sure the new static ip they gave you is a reserved ip address ? If it's not reserved and falls in the range of ip addresses for dhcp you will have problems . Before the move the 2 systems probably where accessing 2 seperate servers for ip call control . Not having access to the network is frustrating at least . I work with some genius network people , I not being one more of a TDM guy learning new technology . If you feel comfortable about it , PM with the info they would need to know and I will answer you with their best solution to your problem .

Man, I don't know what is going to happen if you connect it to a router. But it might just work. LoL
It's not working now so what do you have to loose?
I don't think it will hurt anything to try it.

I guess I'm going back to the days when I was an instructor in a controlled environment where we could put things to the test.

But we also made that equipment do things that the engineers said was impossible !

All kidding aside, I have called a friend that has a great deal of experience with voip and I'll let you know what his opinion is on this monster.

Well this is not a matter of VoIP or routers or such. It's a matter of how the Vodavi system software does the networking between sites as Ed stated in his original post. I find it hard to understand why Vodavi is limited to not having a point to multi-point VoIP network on the same subnet or gateway. If this is the case then you are telling me that the Vodavi product can not do a campus wide school district networking utilizing VoIP. I have a school district now with 10 nodes with the hub and adding three more nodes when school is out. Are you sure you just didn't get some tech in tech support telling you wrong. I think I would escalate this within Vodavi and get to an engineer for clarification. As far as your customer goes, it is not your fault that "their" move precipatated the problem now encountered. If the ultimate solution is acquiring another ISP, then it is your customer's responsibility in that they provide the IP gateway to tie the two systems together. Not yours!

My 2 cent opinion , still says Ed's problem is failing at layer 2/3 . :shrug:

Ed, you know my experience is little with the VoIP networking on Vodavi but, don't you set these up in your shop and test them first?

If so, then aren't you on the same gateway & subnet there? I too find it hard to believe that your problem is because of being on the same gateway.

Are you sure they're not blocking any ports?

If all else fails, erhaps you could get a couple of routers with VPN. Create a VPN tunnel between the sites and change your IP's to what you want.

The guy I called told me he didn't know what was going on either.

Sorry about that Ed. I think I would do like punchdown said and get another guru on the phone at vodavi, if you haven't already done that.

Ok the subnet should not matter.
as for the gateway I could not answer because vodavi must be doing something wrong a gateway is a gateway is a gateway.

I think nfc has a point about the port blocking.
I read recently that the big carriers a required to block some mail ports because of spammers and the like, maybe that is happening here?

I still say it's an IP messaging problem . The server/router sees the IP address but does not know how to route it through call control . My 2 cents again .

I know this sounds crazy but have you tried putting a cheap router at each end and DMZing it? That's weird about the adressing, also if you remove the gateway adress at each site can you still ping the other site like you were on a local network? Cable is wierd like that.

NFC & OBTW might be on track and me not knowing Vodavi, are the TCP & UDP ports forwarded accordingly? Subnet & Gateway should not be the problem you are encountering. Subnet is just another hexidecimal address and Gateway is just an address for the main path. What brand of routers are these sites using? If you say CISCO, there lies the source of the problem because a new rule table has not been written for the new IP address, etc. Do a search on Sundance for BrianVoip and PM or e-mail him for his opinion. He's the best you will ever find knowledge wise on networking and data.

That's just it; we aren't using any routers at all. Both sites were just connected directly to the cable modems at both ends and they worked just fine for over a year. Since they were in different geographical areas of the county, Cox had them residing on different gateways and subnets, so there wasn't a problem. Now that they are across the street is when the problem came along.

We have had a conference call between the director of of Tech Support at Vodavi and a field engineer from Cox on Friday afternoon. They have mutually figured out that this is definitely a limitation on the Cox side and Cox will be contacting me Monday with a solution. I have known this engineer at Cox for several years and I am confident that if he can't figure out a solution, then it simply can't be done.

I was not mentioning onsite routers since you said cable modems in the beginning . Sounds like you have a good tech/engineer helping you now and hope all gets resolved soon !

It sounds like Cox is using Private VLANS, which allow them to isolate users on the same subnet from seeing each other. Cox should be able to configure a group of addresses as part of the same private VLAN, which would allow them to talk to each other.

Since you say the sites are right across the street from each other, it might be worth while to investigate other technologies, such as point-to-point wireless, or free space optical.

If done right, this could also allow them to bridge their LAN's between buildings.

Hope you got a resolution. I agree with the above, with true dual ethernet ports router(s) and proper static routes you should be able to have connectivity. Ed, I posted in the software section a topic on freesco, which would provide a quick bench testing solution.

With private VLANs, it dosen't matter if there is a router or not, the way the network is configured, users can pass traffic to/from the gateway, but not to other users on the same subnet.

The provider should be able to group multiple users into the same private VLAN, but until they do, it's like the other users don't exist.

I can think of a couple possible solutions, but they all involve a third location, and additional equipment at each site. basicly you would use a router at each site to push traffic thru a VPN to a third site on a different network, which would route between the two VPNs.

The best solution would be to either move off the cable modems to a point-to-point link of some sort, thus bypassing Cox entirely, or having Cox group the two sites as part of the same VLAN.

That's exactly what the ISP is going to do. They are creating a VLAN for this customer with 16 different static IP's on each side and separate subnets. All for the low low price of $25.00 per month on each end. Fortunately, this is only a short-term situation until both sites combine into one location, so the money won't be too hard to swallow.

I still think it stinks that the customer has to pay MORE to move their locations closer together!

As always, thanks to all of you for your insight. At least I now have information for the next time this happens.

Ed - I'm curious about this. Could you PM me the last Octet of both addresses and the subnet mask you are using? It's hard to believe that the ISP would tell you it's acceptable that one public address can't talk to another one that they provide. I'm wondering if the problem could be solved by changing the subnet mask (and thereby narrowing the subnet).

I would be delighted to share this with you. It makes no sense to me either. I know very little about this stuff but even I don't buy it.

All that I know is that I conferenced the manufacturer's director of technical support with the best local engineer at Cox Cable that I know of. I monitored the call, heard them talk back and forth and in the end, the Cox guy said he would come up with a solution.

I am equally-curious because we have lots of situations with customers having offices across the street that are paying for P-P T1's. Now that we know of this limitation with Cox Cable, we know to be more careful about suggesting VoIP networking.

From across street to miles away, put up a radio bridge and have no monthly fees.

This is actually not that unreasonable to understand. I had this issue 2 years ago while trying to setup a vpn using Watchguard products. Fortunately for me I was able to have the ISP, change one of the locations to another subnet.

edit:

Just to clarify this not particular to cox or any other ISP for that matter. Its based in the fundementals of "ip addressing" brodcasting (arp request) vs. routing. If anybody wants me to clarify let me know.

~nails

To test this premise that everthing needs to be on the same subnet (which makes no sense to me), hook a router with 2 ethernet interfaces up. Assign the address on one side as the address your service provider gave you assign the other side with a private RFC1918 address. Let me give you a basic config. This assumes that you have multiple static IPs and you assigned 172.16.1.2 to you Vodavi card. I know they say this want work behind NAT but this is a one-to-one it SHOULD work.

interface fa0/0
ip address 8.77.55.1 255.255.255.248
ip nat outside

interface fa0/1
ip address 172.16.1.1 255.255.255.0
ip nat inside

ip nat pool WORD 172.16.1.2 172.16.1.2 netmask 255.255.255.0

ip nat inside source static 8.77.55.2 172.16.1.2

access-list 10 permit 172.16.0.0 0.0.255.255

ip nat inside source list 10 interface f0/0 overload

Can't you re-work one side to a different subnet yourselves? Or because they are public is this not possible?
Also if you are using VLANS then all users must be on the same vlan to see each other unless you have a router to to route to different VLANS, this is confusing with this ISP in there so I'm having a hard time picturing the set up in my mind.
I would suggest putting switches on each side with a router as well somewhere in there and a point to point connection. Then get your own IP block and do all the subnetting you want your way.

Well, it's funny you asked. Our solution had become a requirement for commercial routers at each end, plus more of them for the owner's IP phones at their homes. That was cost-prohibitive and unacceptable for the customer, not to mention the inability to use WIFI devices in public places. I may not have mentioned, but at this time, Vodavi doesn't support NAT, so a VPN would be difficult when working with endpoints outside of the network.

The cable company acknowledged today that there's no way they can accommodate us any other way due to the general plant architecture, so they agreed to let the customer out of their contract. The customer is having DSL installed through Verizon at one site next week. That way, each site will be on a different gateway and this problem will (hopefully) resolved.

It's important to note that if you encounter a customer wanting to network sites together via cable that you make sure that the addresses are on different nodes in the provider's network. If not, the customer needs to be told that routers will be mandatory. We assumed that we could just transfer their "router-less" setup to the new address and we were wrong. After about two months, I am relieved to say that there is some light at the end of the tunnel. The customer is also happy that we are reaching a conclusion.

I am actually very impressed with the people at Cox who have been working with us on this matter because unlike the LEC (un-named), at least they have actually tried to fix the problem instead of just blaming others like "you know who".

I will post back once everything is done to let you all know the final outcome.

Here’s the deal why this will not work.

When a networking device receives a packet for another device, it does basically one of two things. 1) Send out an ARP request or 2) Forward to the gateway. In this case the Vodavi switch received a packet request for a device within the same subnet (subnet does matter), by default it will do nothing other than send out an ARP request (a call out for the mac address associated with the IP address). However the network attached to the Vodavi switch is the internet, and for all intensive purposes the internet disregards these packets since they are layer 2 broadcasts.

Since this is only temp a solution I would have configured one side to DHCP. Around here Cox splits the DHCP clients and Static clients into 2 very distinct subnets. Depending on the Vodavi capabilities there may be a setting that allows for the DHCP side to initiate contact with the static side and update any address changes (many VPN devices support this feature now). If not, you could manually enter the destination of the static side to the address given by the DHCP side. From experience, I have clients that use DHCP and rarely have to change settings, many times a DHCP address will not change for months.

Vodavi doesn't support NAT? Wouldn't your routers do that if you had any?

DW:

From what I have been told by Vodavi, the answer is no. We have never had success getting their system networking to function behind routers. We have always installed a small switch ahead of the customer's router and relied upon static IP's. It worked very well for years until this site on the same carrier node came into play.

NAT is in the works with Vodavi, but as of today, it has not been accomplished. Don't ask me why, I have very little understanding of IP. All I know is what I am learning along the way, but even the cable company (COX) has admitted that without a significant investment in routers for the multiple sites, it can't be done.

The whole problem here is that the customer had a perfectly working IP setup for over a year, so they don't feel that they should have to spend $2K for routers and for various IT guys to get involved when all they did is move to a new site across the street. I can't say that I disagree.

Does not matter if you invest in router or not. If you have router A with the same address as Vodavi A and router B with the same address as vodavi B, you still have the same issue as i explained it above. Routers route... that means it sends packets between subnets. You have two IPs in the same subnet.

Oh, I am not doubting what you said, Rusty, it's just that the customer isn't willing to pay for the routers and installation/programming cost. I agree that if they were in place, something as simple as just placing the IP's in the DMZ would work. It's just that; they customer doesn't think that they should have to pay for all of this additional hardware just because one of their locations relocated. Fortunately, the cable company agrees and by going with DSL at one site, they will actually be paying less per month.

I really do appreciate your insight though. Even though we think we have reached a "poor mans" conclusion, your information has been helpful to me for future applications.

Well,

I just wanted to clarify that simply adding routers will not solve the problem, unless you are able to change one side to another subnet, and if that were the case this thread would be non-existent.

However I would say Zyxel vpn routers with one configured as DHCP (giving you that different subnet)would only cost about $250 per location.

But yeah, ultimately if the customer does not want to pay then they dont want to pay.

Yes, I know Rustynails. We just couldn't get a separate subnet while being on the same node's gateway. It has been confirmed by Cox that they just don't have the ability to do this due to the inherent nature of the US cable distribution network.

Ironically, we tried one, then two Zyxel routers (provided by Cox) and still, they couldn't get it to work. I don't understand why. If they say it can't be done, they are the service provider, they suggest DSL and they let the customer out of the contract, then either it really can't be done or they just don't know how to do it.

Regardless, $250.00 times four locations on the VPN for now would have meant $1,000.00 to restore something that the customer already had. That cost is before installation and programming labor for them. The worst thing is that this is just a temporary situation for 3-6 months as the customer consolidates locations, so a hardware investment of that magnitude is out of the question. I don't blame them.

Ed, the key is setting one location to DHCP, where I live, Cox always provides a different subnet for DHCP request distinctively seperate from their stattic addressing, regardless if you are on the same node.

At that point you allow for the DHCP router to initiate the VPN connection.

And to be quite frank with you, i think Cox tech/engineers are horrible. Think about it they went from CATV to internet and dialtone in just a matter of years. A lot of crash and burning at our expense. However as i have mentioned in another post, once you get their service to your facility, up and running, i think its the best out there; price and quality.

Well, several people requested that I post back with the outcome, so here it goes:

The cable TV provider let the customer out of their contract for one site. They admitted that they knew what the customer needed and simply couldn't provide it.

We had the customer order DSL from Verizon for the smaller of the two sites. At first, there was a minor problem since they were using AT&T as their CLEC. Eventually, Verizon brought the DSL in on a separate cable pair on a separate account. Two days late, but they got it there.

Then, it turned out that they set the account up for PPOE, as in for a typical residential user who only uses their DSL for Internet access. We contacted them to change it to provide us with static IP addresses, but it took them eight business days to make it happen. They would not provide us with any technical support on their modem; that we would have to reconfigure these settings on our own through their technical support website. This was nothing but a PDF of the manual for the modem.

Once this was done, the process resulted in a new batch of IP addresses being issued, but they didn't tell us that they changed, so we chased our tails and kept trying to make the original ones work. Eventually, through the grace of God, we got in touch with someone at Verizon with more than a pulse who figured this out. He remotely reconfigured the modem for us in about two minutes. Our technicians literally watched the "sync" lights come up on the VoIP cards while he was on the phone with them.

Two weeks after the DSL was installed, it is finally working. Months of wasted time, and countless dollars down the drain, but at least the systems are working properly and the customer is happy.

The customer hand-delivered a check for the second half of the system relocation and one invoice for our time spent on-site troubleshooting the IP situation. We'll probably never recover payment for the dozens of hours spent on this matter, but at this point, I am glad to see it resolved.

Now, here's the punch line. The customer will be closing one site within three months and consolidating them in one building. :bang:

I wanted to wait at least a week before posting back about this matter just to make sure it remains working. So far, so good. Thanks to all of you for your ideas and assistance.

Dear ev607797,


You wrote:

"Now everything is 24.XXX.XXX.XXX at both ends with the same subnet."

Just because both ends have an address of 24.xxx.xxx.xx does not mean they are on the same subnet. You need the entire IP address and SUBNET MASK of each system to tell if they are on the same subnet.

You also wrote:

"We do have different static IP's for each network card. It's the fact that these IP's are in the same subnet that is apparently causing the problem."

This is very unlikely to be the problem. The most likely problem is that the configuration of both systems needs to be changed to reflect the new IP address and other network information. In particular the security information needs to be changed and any security certificate needs to be updated.

If you have solved this problem, I would like to know how. Did switching to a different internet service provider solve the problem?

Yes, it was resolved on July 5. I detailed what we did in my post right before yours. Thanks for checking in anyway.