DOCSIS 3.0 Partial Service
DOCSIS 3.0 Partial Service is a new term encountered in the DOCSIS 3.0 MULPI specification and realized in field deployments of DOCSIS 3.0 cable modems using upstream bonding. This was a topic that I touched on in this years SCTE Cable-Tec Expo, but will explore in greater detail in this article. Partial service can be considered a feature because the cable modem will stay online even when one or more upstream transmit channels goes offline. A related topic to partial service is “impaired service”, but I will cover that in a related article.
Defined
A modem is in a partial service mode of operation any time it is operating with a subset of the channels in the Receive Channel Set (RCS) and/or Transmit Channel Set (TCS) because a channel has become unusable, either due to an inability to acquire a channel or because communication on a channel was lost during normal operation.
A typical bonded upstream service group will have up to four channels, that when aggregated together are capable of transmitting over 100 Mbps at 64-QAM modulation in a 6.4 MHz bandwidth. If only one upstream channel is out of service in the “partial service” scenario, it is possible that the end-user will not notice any impact to performance. A three channel DOCSIS 3.0 modem would still be capable of over 80 Mbps. Therefore, in a DOCSIS 3.0 network, partial service is:
- Hard to detect after registration
- Most current test instruments will not identify it
- A throughput test may not identify it on a loaded plant
Two scenarios for partial service:
- The channel was down when the modem came online
- Test equipment will show fewer channels bonded than expected
- Will likely not cause any noticeable performance issues, unless the service tier is very close to the maximum throughput performance with a full channel set
- The Errors for the channels that could not be acquired are reported to the CMTS in the REG-ACK
- The channel becomes un-usable following TCC (Transmit Channel Configuration)
- The channel is already in the bonding group for this modem, and has become unusable. The CMTS will stop granting transmit opportunities on this channel until communication is reestablished via the ranging process. CMTS’s may handle re-acquisition differently and may not re-acquire at all until the CM is rebooted! (it is a SHOULD requirement).
- Will cause temporary performance issues between the time that the channel becomes unusable and the time that the CMTS realizes that it is unusable (Range interval + Range timeout or about one minute). Following that time, the CMTS will refrain from granting timeslots on this channel, and performance degradation will stop.
Troubleshooting
When a technician is troubleshooting, partial service is one of the issues that can be experienced. It is unlikely that most technicians would go to the field specifically to troubleshoot partial service, as it is not particularly noticeable unless the end-user is bumping up against the max transmit performance of the complete TCS.
The first indication to the technician that a portion of network is experiencing partial service is when the test instrument only bonds a portion of the Transmit Channel Set (TCS). As seen in the screen captures below, the technician expects to see 4 channels in the TCS (as in the first screen shot), but the instrument only reports that there are 3 channels bonded (as in the second screen shot).
In this situation, the CMTS is aware of the partial service, and would refrain from granting MAP opportunities on the channel that is unavailable. Therefore, unless the service level of the end-user is above the capabilities of the 3 bonded channels (80+ Mb/s), the end-user would likely not experience any degradation of service. (assuming that the other three channels are not experiencing performance issues)
When the technician determines that a partial service situation exists, the goal is to determine what is causing the partial service. To accomplish this, the technician should begin traversing the network toward the CMTS, testing at each test point (Ground Block, Tap, Amps, Node) until the point where the missing channel(s) return to service.
When the missing channel(s) have returned to service, it could mean one of two things:
- The gross network issue has been passed, and the previously missing channel is in service and operating at peak performance.
- A minor network issue has been passed, allowing the previously missing channel to return to service, but other network issues continue to impair the performance of the channel.
Because both of these results are possible, it is very important to take a closer look at all of the upstream channels once a partial service scenario is fixed. The sections that follow will discuss the process of isolating individual upstream channels in a bonded environment to determine performance of each.
Summary
Partial Service can be a tremendous advantage for cable operators because a DOCSIS 3.0 cable modem will stay online even if one or more of the upstream bonded channels goes offline due to RF impairments. The subscriber will not likely notice any service disruption with the exception of a short disruption of about 15 seconds when the CMTS and cable are attempting to re-range on the impaired channel. The subscriber may also notice throughput issues if the DOCSIS upstream is in an over-utilized scenario and they have fewer upstream channels to use based upon their service tier.
A DOCSIS Partial Service outage should not be considered a permanent band-aid to resolve RF plant issues, but a temporary measure to keep the subscriber online while the impairment is identified and resolved.
Upcoming events can be seen under Broadband Events. Previous events can be seen under the blog.
- If you are watching this on youtube please hit the subscribe button!
- Let us know what you think and remember to share!
- You can find slides at the bottom of the page and some on slideshare.
- Find out about events or articles by following us on Twitter, LinkedIn or Facebook too.
Also available on iTunes, Google Podcasts, Spotify, vurbl see podcasts “get your tech on”.
We have been trying to understand this US Partial mode for a long time. It is so hard to find answers. Thank you very much for your very helpful information
Mr Volp can you tell us what US status “dr” means?
Hi Jon,
“Brady” is fine. Can you elaborate more on the “US status dr”? What CMTS are you using?
There is a Cisco init(dr) state which means DHCP Request, which would be a message sent by the CM to the CMTS during DHCP, but this is after the DHCP Discover message was already sent by the CM and the DHCP Offer received from the DS.
-Brady
Brady
Yes we have a Cisco CMTS. Here is what we see and it usually clears with a reboot of the modem.
Verbose command
Partial-Mode Information : reason 0x1 failed-tcs 0x6
Extended Upstream Transmit Power : 0dB
Upstream Channel : US0 US1 US2
Ranging Status : sta dr dr
Upstream SNR (dB) : 31.46 33.1 33.97
CNNW10012A#scm partial
MAC Address IP Address I/F MAC Prim RCC UP-reason/ State Sid ID Failed-tcs
001d.d0b4.b4bb 10.101.192.191 C5/0/1/p w-online(pt) 4791 1 0x1 / 0x6
Ranging status dr
Hi Jon,
I will contact you offline.
-Brady
Dear Mr Volpe,
Can you please confirm that the Max. modem transmit power does not change when f.i. 2 out of 4 US channels are in partial mode?
A lot of people I talked too, mentioned that the Max TX-level can be 3dB higher because there are only 2 channels active.
I disagree with them :-), because the modems is still active in a 4 channel bounding group
Regards
Bram
Hi Bram,
Depending upon how the CMTS is configured it is possible that you and your colleagues are correct.
The CMTS has several options. The default option, in which case you are correct, is to continue to have the cable modem operate in partial mode. In this case the modem will continue to range with the upstream channel that is offline. The remaining online upstream channels will not change power levels because as you say they are still part of the same bonding group. So everything remains constant.
If the CMTS is such programmed / configured (refer to your vendor), the CMTS is permitted per the DOCSIS standard to remove a faulty upstream channel from the channel set, in which case the CM would no longer be in an errored state. The CM would also be operating with less upstream channels and would therefore be able to transmit with a higher upstream transmit power. I have not seen this occur in an actual production environment, but it is permitted by DOCSIS.
Finally, the CMTS could move the CM to another upstream bonding group if it is available. This is likely not a typical case, but suppose one had load balancing between two sets of upstream bonded channels (fiber nodes), for example. One set had three bonded upstreams and the other had two bonded upstreams. If the modem was on the three bonded upstream fiber node and one upstream failed, then the CMTS moved it to the other fiber node (virtual) with two bonded channels the CM would have a higher transmit power. Again, this is not a very practical use case, but allowed in DOCSIS.
In summary, the most likely case is the first one where the modem remains in partial mode with the same transmit power. However it should be noted that DOCSIS does allow for other modes where the modem could transmit at higher levels.
-Brady
Hello Brady,
Thank you for the informative article – You provide a tremendous service to the community, and it is much appreciated!
We have some Cisco 3010 modems which appear to be misbehaving – They will range, come online, everything will appear to be fine (modem is pingable, subscriber can pass data) and about 90 seconds later, the modem stops passing data. Behind the scenes, when I run a debug cable mac-address [modem] and debug cable dhcp on the uBR10K, I see that the modem cycles through the DHCP process about 10 times … Odd. When I issue a show cable modem verbose, I am presented with “CM Initialization Reason : TCS_FAILED_ON_ALL_US”
Based on your article, it would appear that the modem has gone into partial service, however, if it is failing on ALL upstreams, am I correct in saying that there is no way it could transmit on the cable network?
Interestingly, the CM continues to show online after it makes it’s initial registration. Though it will no longer pass any data. I am thinking it may be a firmware issue, as I can plug in an identical modem running DPC3010-v302r12901-110516a-COX..2.3.0_R1..185933. and it works fine. The misbehaving CM I have been able to isolate is running dpc3010-v302r12901-091119a..2.3.0_R1..185933.
Have you seen this in your travels? It is causing our service desk and field techs a great deal of consternation …
Thank you as always!
Joe in Okinawa
Hi Joe,
Great to hear from you again. I will reply offline as
-Brady
Hello Brady
This is a great site. Thanks for all the info.
I’m having a provisioning issue where the CM sometimes get’s online sometimes doesn’t. It fails while in ranging, the log shows “TCC Acquisition failed : TCS_FAILED_ON_ALL_US”.
Any idea on might be causing this?
Best Regards
Willy
Hi Willy,
Your question was addressed in the G+Hangout which you can watch here or download on iTunes. https://volpefirm.com/load-balancing-docsis-3-1-episode-7/
Thanks,
Mia
Hi Brady,
I have been reading your topics from this site with interest, and I was struggeling with a question where we would make Partial Service more visible to the operators. We are using SNMP based systems and I am looking for any DOCSIS standard MIBs that would provide visibility for Upstream and Downstream Partial Service conditions.
For Upstream I have figured out a solution, where I can retrieve the UpstreamRangingStatus using the following MIB:
docsIf3CmtsCmUsStatusRangingStatus 1.3.6.1.4.1.4491.2.1.20.1.4.1.12
Modems that provide any other status then success (4) will be marked with Partial Service.
I would like to have a similar solution for Downstream, where we could observe these modems and get an overview of modems with Partial Service in the Downstream direction. It would be even better if it is possible to get an overview of Channels that are deleted from the DBG?
Kind regards,
Edwin Heuser
Hi Edwin,
This is a more detailed answer. You might also try https://volpefirm.com/load-balancing-continued-episode-8/
Hi Brady
i still dint see the answer what dr means in ranging status: and aldo there is another one (di)
UBR10k-PRE4(config-if)#do scm 0880.39e6.7300 ver | i Rang|Up
Extended Upstream Transmit Power : 0dB
Upstream Channel : US0 US1 US2 US3
Ranging Status : dr sta sta sta
Hi Asian,
dhcp process is “dora”. discover, offer, request, and acknowledge. we have init(d), init(io), init(dr) and init(i) to indicate those 4 states of registration.
In regards to US states, this means down with ranging enabled. Any state other than STA is bad and mode will be marked /p for US bonding partial mode.
Also see: https://volpefirm.com/cablelabs-f2f-and-acta-along-with-other-topics/
actually i found it here is the good reference :
* IM = waiting for initial maintenance
* SM = waiting for station maintenance
* Cont = continuous ranging (ranging miss or adjustment is needed)
* STA = station maintenance
* DR = down & recovery with extended ranging opportunities
* DT = down due to timeout, and no recovery
* DI = down due to interface shutdown
Thanks
Hi, Brady,
What are the primary reasons if the US partial mode do not go back to UB after some recovery?
Hi Maria,
CMST setting; (Cisco CMTS default is cont ranging on failure to allow it to come back automatically). Some customers have turned this off.; US MER is still bad; time offsets are out of whack and or map advance is too aggressive. Also, this question was discussed in our last G+HOA. You can either watch the video or listen to the podcast here https://volpefirm.com/cablelabs-interop-docsis-3-1-channel-bonding/
I am experiencing an issue migrating a property from hardline coax to fiber backbone the CM’s are DOCSIS 3.0 and bonding only on one of two upstream channels. The channel that fails is registering 10db difference (lower) in power level. Is there a common reason for this?
Thanks
Hi Kevin,
2-ch US bonding using 64-QAM maxes out at 54 dBmV, so it may be an US level issue.
My modem experiences upstream partial service – only 1 out of 3 channels is good.
This is with Comcast in zip code 06801 (CT).
First comcast suggested it was a backlevel (End of Life) modem problem…. But now I’ve tested with three different model modems and see the same problem. DPC3000, SB6141 and now CM820A.
I see continual timeouts T3 and T4 every 30 seconds in the modem logs.
Today, tech over phone suggested removing a splitter.
When I do that, no more timeouts – But I still don’t think all upstream are working because
status message shows “—-” empty instead of real numbers for power and upstream symbol rates.
Tech will make a home visit on Wednesday…
Advice?
Thanks!
hello Brady, I have a question why I am getting partial service on CMTS as a bad TCS, but in cable modem a got signal. Is there any Feature as a CMTS level that can face this?
Hi Ariel,
Great to hear from you.
I cover this exact question in this article: https://volpefirm.com/docsis-3-0-partial-service/
This is also a very common condition where your cable modem has RF signal, a hand-held meter shows the signal as good from an SNR/MER and power standpoint, but still the modem will not use the channel. So what happens? The technician will replace the modem, but still partial mode exists. But why? Every-time we find that the root cause is always micro-reflections or group delay which are not discoverable by a test meter. So what is the solution? Well, we have had this conversation before – it is PNM. PNM gives you visibility into hidden impairments like micro-reflections and group delay.
You can create new TCS groups in the CMTS so that the modem will bond to 2 channels or even 1 channel. This will make your modem no longer show up as partial-mode, however your subscriber will now have slower upstream speeds. This is not a good solution for obvious reasons.
My recommendation is to use NimbleThis PNM. We have many good partners as well and would love to help you. 🙂
Take care and hope we can one day help you Ariel.