Tags Posts tagged with "OSS"


2 387

In continuation to my previous blog on Network Discovery and Analytics, which mostly revolved around  some of the core but basic functionality that a Discovery system needs to possess. In this post lets discuss some of the key challenges that discovery systems have to face when they first get deployed.

1)      Resistance from network operations teams.

  • Network operations for example would like for the network not to be disturbed during peak traffic hours or during maintenance windows. Towards that intent, the Network operations teams would like to be confident that a discovery system can be set up such that the system will at no cost touch the network for a configured duration. The network operations team would expect to have confidence in the discovery system that irrespective of the stage at which a network polling activity is in, the activity needs to be suspended on the event of hitting a black out window.
  • Non – intrusive discovery is what a network operations team would be confident about. By non-intrusive discovery, I mean that a device which is already overloaded with respect to its resource consumption should not be further affected by querying for information to perform a discovery

2)      Resistance from IT teams managing N/EMS systems.

  • In the event of discovery from NBI of N/EMS or from gateways that maintain stateful sessions, in a lot of situations it would be required of the discovery system to not consume more than a configured number of sessions. This may be required to make sure that other northbound OSS systems a not denied a session request from the N/EMS or gateways. It may also be because only a few NBI sessions have been purchased from the equipment vendor and using more number of sessions than purchased may lead to a contract violation or error in discovery.

The OSS Network discovery applications have to evolve and stand up against the above challenges by introducing functionality like blackout support and network hit throttling capability.

Moving forward to the next stage of the evolution. Tier-1 or Tier-2 operators have discovery systems deployed and use this information to keep inventories in sych with the real world. But inventories typically store services or end-to-end circuits. And what is discovered are individual components of a service or circuit which have been provisioned within a network element. It is important from the perspective of the service provider to view the current state of a circuit in the Network, compare it with the inventory to fix misalignment. This brings in the responsibility within the Discovery system to be able to assimilate service components discovered within each network element along with the network element interconnectivity information and be able to plot end to end circuits for various technologies.

As of today,  a few tier 1 operators who are pretty mature in their process are looking towards evolving the discovery system’s capabilities to be in near real time synch with the state of the network. A system which is able to listen to traps/events from the network and refresh its database with the latest state of the network. And use the near real time discovery system to evolve the inventories to be near real time up-to-date with the reality in the network.

It’s another thing that a lot of inventories even today are far from accurate despite a whole lot of tools out there in the market specifically designed to solve data integrity issues of Inventories. And the reason for that is the lack of a sound practice around the usage of these tools or a lack of commitment to adhere to a data integrity program or a failed OSS inventory transformation project etc

In the interim, while operators are trying to get their inventories cleaned up we believe that the Discovery system along with intelligent and actionable analytics has a lot more to offer to the planning and service assurance teams which has been mentioned below.

1)      The discovered physical fabric can be compared and/or enriched with OSS inventories and ERP systems to build a 3600 view the asset. The 3600 view when recoded for a period of time becomes a very power source to a bunch of analytical function which would provide actionable intelligence to planning and network operation teams in helping with Capex avoidance.

2)      The discovered logical/service component information when captured for a period of time can again be a source to bunch of other analytical functions (Time series trending and forecasting, What-If modeling and optimization) to help network operations and planning teams to perform network capacity management on accurate and as discovered information.

3)      The discovered information when assimilated with topological analytics to calculate end-to-end circuits/services can be a powerful source of information to Service assurance teams to overlay alarms and generate a service impact view in their NOC.

0 654

Let us look at why OSS Network discovery systems came into existence, how they evolved over years and in our opinion the areas in which it would play a critical role going forward.

Most Tier-1 and Tier-2 service providers have at some point in time invested in building one or more inventory system which would work as a central repository for functions like Planning, Design, Fulfillment and Assurance. For these functions to have a reliable and up-to-date view of the network, it is important that depth and breadth of information is captured by the inventory. By breadth, I mean the ability to store information from multiple device types from multiple device vendors. And by depth I mean the ability to store physical inventory and also logical inventory for multiple technologies like PDH, SDH/Sonet, DWDM, Ethernet, ATM, FR, IP, MPLS, xDSL, FTTx, 2G, 3G, LTE etc. These Inventories have largely been maintained/updated by manual data entry and in the event of weak process or lack of discipline in following processes; these systems would very quickly go out of synch with the network. And hence OSS Network discovery systems first came into existence in order to help keep inventories up-to-date with the real world. Towards this intent, OSS Discovery systems needed to take care of the following requirements.

1) Data that needs to be acquired from the network

  • Ability to perform shallow discovery i.e. discovery of physical fabric (Physical inventory like Racks, Shelves, Cards, Ports) from most leading-class networking equipment vendors based on standards like SNMP Entity Mib-2, TMF 814 etc.
  • Ability to discover deep logical information like vlans, ip interfaces, ip subnets etc again using standard mibs (like RFC 2233, RFC 2674), MTNM etc.
  • Ability to discover inter-connectivity (PTOPO) between devices wherever possible

2) How data needs to be acquired from the network

  • A lot of device vendors equipment cannot be discovered using standard SNMP interfaces and hence the need to be able to discover using other network management protocols like TL1, MML, CMIP, Corba, WebServices.
  • Many a times Network operations teams are not comfortable with multiple IT systems (N/EMS, Discovery, Fulfillment, Assurance etc) connecting directly to the devices for multiple reasons like worried about resource consumption on devices which may affect revenue generating traffic and access control list and login configurations on all devices need to be modified for discovery servers to access the device. This may seem like a trivial one time activity, but a few large Tier 1 operators which have strong process in place; this activity can run into a few months.

And hence discovery systems were forced to interface with north bound interfaces of N/EMS systems or gateways in order to perform discovery. These NBI APIs on a lot of occasions end up being propriety or customizations done on standard specifications.

For reasons mentioned above, discovery systems with ability in capturing breadth and depth in discovery have had to build a lot of device vendor specific adaptors instead of standards based adaptors.

1 48

Now, referring to the title, you may be thinking: That’s a rather cheeky thing to say given the high direct and indirect costs of errant data incurred by virtually all operators.   You might cite the significant Opex penalty related to reworking designs and to service activation fallout.   I get that.  What about the millions of USD in stranded Capex most operators have in their networks?  Check.  My personal favorite comes from Larry English, a leading expert on information quality, who has ranked poor quality information as the second biggest threat to mankind after global warming.  And here I was worried about a looming global economic collapse!

My point is actually that the discrepancies themselves have no business value.   They are simply an indicator of things gone bad.  The canary in the coal mine.    These “things” are likely some combination of people, processes and system transactions, of course.  Yet many operators make finding and reporting discrepancies the primary focus of their data quality efforts.  Let’s face it, anyone with modest Excel skills can bash two data sets together with MATCH and VLOOKUP functions  and bask in the glow of everything that doesn’t line up.  Sound familiar?

For context, I am mostly referring to mismatches between the network and how the network is represented in back-office systems like Inventory—but the observations I will share can be applied to other domains.   Data anomalies, for example, are all too common when attempting to align subscriber orders and billing records in the Revenue Assurance domain.

Too often, Data Integrity Management (DIM) programs start with gusto and end with a fizzle, placed on a shelf so that shinier (and easier!) objects can be chased.  Why is this?  Understanding that I am now on the spot to answer my own rhetorical question, let me give it a go.

  • The scourge of false positives: There are few things as frustrating as chasing one’s tail.  Yet that is the feeling when you find that a high percentage of your “discrepancies” are not material discrepancies (i.e. an object in the Network but not in Inventory) but simply mismatches in naming conventions.   A DIM solution must profile and normalize the data that are compared so as not to spew out a lot of noise.
  •  The allure of objects in the mirror that are closer than they appear:  OK, not sure this aphorism works but I trust you to hang with me.   I am referring to misplaced priorities— paying attention to one (closer, easier) set of discrepancies while ignoring another set that might yield a bigger business impact once corrected.    Data quality issues must be prioritized, with priorities established based upon clear and measurable KPI targets.  If you wish to move the needle on service activation fallout rates, for example, you need to understand the underlying root causes and be deliberate about going after those for correction.  Clearly, you should not place as much value on finding ‘stranded” common equipment cards as on recovering high-value optics that can be provisioned for new services.
  • The tyranny of haphazard correction: I’m alluding here to the process and discipline of DIM.  Filtered and prioritized discrepancies should be wrapped with workflow and case management in a repeatable and efficient manner.  The goals are to reduce the cost and time related to correction of data quality issues.  If data cleanse activities are unstructured and not monitored by rigorous reporting, the business targets for your DIM program are unlikely to be met.
  • The failure to toot one’s own horn: Let’s say that your data integrity efforts have met with some success.  Do you have precise measurements of that success?  What is the value of recovered assets?  How many hours have been saved in reduced truck rolls related to on-demand audits?  Have order cycle times improved?  By how much?   Ideally, can you show how your DIM program has improved metrics that appear on the enterprise scorecard?   It is critical that the business stakeholders and the executive team have visibility to the value returned by the DIM program.  Not only does this enable continued funding but it could set the stage for “self-funding” using a portion of the cost savings.
  • The bane of “one and done”:  For a DIM program to succeed in the long run, I suggest drawing from forensic science and tracing bad data to underlying pathologies… i.e. people, process and/or system breakdowns.   A formal data governance program that harnesses analytics to spotlight these breakdowns and foster preventive measures is highly recommended. The true power of DIM is in prevention of future data issues so that the current efforts to cleanse data will not simply be erased by the passage of time.

Identifying data discrepancies is a good first step.  Correcting and preventing them is even better.    Institutionalizing DIM via continuously measuring and reporting your successes… well, you get the idea.

Follow Us