1. Purpose of Proposal
2. Background
3. Needs Analysis
4. Potential Solutions
5. Test and Results
6. Appropriateness
7. Evaluation
8. Conclusions & Recommendations
This paper is an explanation of the issues, problems and choices surrounding the integration of NetWare with the campus backbone network. Data Communication and Network Services (DCNS) has the goal of enabling departments to use and share NetWare resources easily and seamlessly across physical network boundaries. The specific implementation or solution is not as clear as the need for one. This paper seeks to define and describe this need, the potential solutions, and to recommend actions DCNS can take to achieve the above goal.
The construction of the current UC Berkeley campus network began in 1984 and is still underway. The campus network is growing by roughly 3000 connections per year and currently includes approximately 16,000 computers connected to 140 subnetworks. Most subnetworks are 10 Mb/s Ethernets and are connected to the backbone via 35 Proteon CNX500 routers and 17 P4200 routers. The campus backbone network consists of two FDDI rings. One backbone subnetwork is dedicated to the Department of Electrical Engineering and Computer Science (EECS); the other is for the rest of the campus. These rings are joined by a Proteon CNX500 router .
NetWare servers began appearing on campus in the late 1980's, usually in administrative units not yet connected to the campus network or in other units with a degree of independence from central campus computing plans. Information Systems and Technology (IST) was not involved in the purchase, installation and support of NetWare until 1990 when the Workstation Software Support Group (WSSG) negotiated a campus site license with Novell, Inc. WSSG began using NetWare internally in their microcomputer facilities. WSSG supports departmental NetWare system administrators by mediating a users group, negotiating volume purchase of Novell products, establishing direct communication with Novell engineers, providing consultation and presenting special workshops. There are over 70 NetWare servers on campus accessed from approximately 2000 DOS and/or Macintosh clients by more than 15,000 users.
DCNS maintains a policy of routing only a single, non-proprietary network protocol across the campus backbone. All campus routers are configured to route and forward TCP/IP packets only. IPX-based communication (and, therefore, departmental NetWare servers) is isolated on local subnetworks. DCNS recently began contributing to the design and management issues of IPX (Internet Packet eXchange) networks by suggesting standards for NetWare Ethernet frame types and IPX network numbers.
Included with NetWare v3.11, released on campus in mid 1991, was support for TCP/IP and a NetWare Loadable Module (NLM) named IPtunnel.NLM. This NLM allows two NetWare servers, or tunnel peers, to encapsulate IPX packets and transmit them across an IP-only backbone. Several departments, including WSSG, began implementing IPtunnels as a way to link NetWare servers and users isolated from each other by the campus backbone routers. As additional tunnels were added, (sometimes by two departments sharing a single Ethernet and tunneling to different places which in turn tunneled to yet other networks, and so on), a very complex IPX internetwork was created with little or no consideration of physical network design. These tunnels are notoriously difficult to maintain and offer less than optimal throughput.
The major disadvantages of the current situation are isolated NetWare LANs, inadequate performance provided by IPtunnels, difficulty in configuring and troubleshooting IPtunnels, and a non-rational IPX internetwork design.
Novell recently released the NetWare/IP product which allows NetWare services to operate on the TCP/IP protocol suite rather than the IPX protocol suite. DCNS has also obtained the IPX support module for the Proteon CNX500 routers. Both of these solutions allow NetWare resources to be shared across the campus network -- only NetWare/IP complies with the single protocol standard for the campus backbone.
The next section will describe the typical ways NetWare is used on campus, examine three departments in detail and will then focus on two specific solutions.
It is important to note that NetWare is not a general purpose operating system. It does not, for example, provide CPU service to users in the same way as UNIX. Users login to NetWare servers to authenticate their access to file systems and other resources. The client computer from which a user is accessing the NetWare server provides CPU service. NetWare servers are particularly well suited to providing print and file services or other services based fundamentally on these types of services, e.g., tape backup or LAN-based email.
With the purchase of add-on products, NetWare servers can provide file and print services in environments native to platforms other than DOS/Windows. For example, NetWare file systems can be made available to Macintosh systems via AppleTalk and to UNIX systems via NFS. However, NetWare does not provide CPU service to any platform -- while different platforms may share data via a NetWare server, the applications that manipulate that data, wherever located, are specific to that platform/OS.
While NetWare servers are typically set up to meet the shared file and print service needs of individual departments, there are several instances where access to NetWare resources must cross the campus backbone:
3.1. Departments with Multiple Subnets
Several departments cross multiple subnetworks. These departments are either very large, (e.g., College of Chemistry), dispersed across campus, (e.g., WSSG Workstation and Microcomputer Facilities), or both, (e.g., the Library.) In these situations, use of NetWare services ranges from the occasional need to reach a remote server for administration or monitoring to complete sharing of software applications. (These three departmental scenarios will be considered in greater detail below.)
3.2. Remote Clients
The department has a small number of remote users not located in the same building, i.e., not on the same subnet as the rest of the department. Those remote users need as much access and performance (throughput) to departmental applications and data as local users.
3.3. Campus information service
The department runs an information service used by many other departments, e.g., the Berkeley Equipment Tracking System (BETS) maintained by Equipment Management. Users of these services would likely need access to centralized data but could have the application available locally.
3.4. NetWare from Home.
DCNS recently began providing Home IP service via SLIP and/or PPP. NetWare users are likely to want access to their departmental NetWare servers from home.
College of Chemistry
The college currently has five subnets. NetWare clients are distributed throughout the five subnets but only two of subnets have NetWare servers. To allow client-server access for all five subnets they are connected by a private, internal IPX backbone. Each server has a second network adapter connecting it to the IPX backbone. A single-user NetWare server with multiple network adapters is connected to the other three subnets and the IPX backbone. Through this internal backbone, the college has integrated their NetWare servers and clients. It has been an adequate solution for the college's needs. However, it is non-standard and will not meet future needs when the college adds an additional three subnets in Tan Hall (Chemistry Unit III).
Microcomputer Facilities
Workstation Support Services' Workstation and Microcomputer Facilities (W&MF) operates nine computer facilities or labs with over 350 PC and Macintosh clients and seven NetWare servers. They provide hands-on instructional facilities to 150 classes and over 9000 student users. These facilities are the only open, general-access computer labs on campus. For the most part, with nearly a server at every lab site, each of these facilities is wholly independent. Servers are monitored remotely through IPtunnels by W&MF staff. The W&MF group would like to halve the number of servers they support, provided throughput across the campus backbone would be sufficient for launching applications. Because of their overhead IPtunnels do not provide sufficient throughput for this scenario. If it was possible to consolidate NetWare file services they would implement print servers to meet local printing needs.
The Library
With dozens of branch libraries, as well as the Main and Undergraduate libraries, the Library uses (or shares) 26 different subnets. (There are also four subnets planned for the Doe Addition). The Library currently has over 400 users, over 218 PC clients, six NetWare Servers and three Meridian CD-Net towers. (Additional CD-Net Towers are planned in cooperation with other departments.)
Currently, each CD server must be located on the same subnet as a NetWare server. As dedicated DOS clients, the CD servers are not capable of running the IPtunnel.NLM. They were tested with IPtunnel.exe and did not function properly. They have not been tested with NetWare/IP. Currently, the only way they can be accessed by remote clients is if their IPX traffic is tunneled through nearby NetWare servers.
All branch library subnets have print servers, either dedicated IPtunnel.exe clients running a print server application or printers connected directly to the network via HP JetDirect devices. Client workstations on networks without nearby servers are also IPtunnel.exe clients.
The Library Systems Office staff would like to reduce the number of NetWare servers (and server licenses) that they currently maintain. They would like to implement a single, centralized tape backup system for all NetWare servers, wherever located. This system will work with IPtunnels but has not been tested with NetWare/IP. (Backing up NetWare servers across routers could significantly add to network traffic and should be scheduled during off-peak hours.) The Systems Office would like to locate CD servers independently of NetWare server locations and provide equal access to all services from all clients, local and remote. In addition, if a campus-wide NetWare internetwork were constructed, Library CD servers could be accessed from student computer labs and faculty offices. Currently the Library is suffering all the troubles of IPtunnels and its NetWare resources are inconsistently integrated.
These departments each have internal integration needs; they, and other departments, also have inter-departmental integration needs. For example, there currently is no central file server for software distribution for NetWare servers and DOS/Windows clients that can be reached via file transfer commands native to those platforms. Currently system administrators and users must use FTP to transfer files from tuna, the WSSG software distribution file server for DOS/Windows platforms. A solution that integrated NetWare services across the campus backbone would allow this service to function natively for DOS and Windows clients. In fact, a recent and successful experiment using NetWare/IP clients, allowed NetWare system administrators to perform server upgrades across the campus backbone from HOOK, a WSSG NetWare server.
There are a number of possible options for integrating NetWare services with the campus backbone; two are worth looking into in detail: routing IPX (installing and maintaining IPX modules in campus routers, routing IPX packets natively on the backbone), and installing and maintaining NetWare/IP (Novell's product that allows NetWare to operate on IP-based network and transport layers.)
4.1. Routing IPX
4.1.1. Description
This option involves routing IPX packets natively on the backbone. It would require installing and maintaining IPX modules in all campus routers which could then configured to accept and forward IPX packets. Routers would be configured with IP addresses and IPX network numbers all interfaces. In addition to maintaining routing tables of IP networks, the routers would also maintain tables of NetWare services and IPX network routes.
The effects on campus network routers and traffic would be a result of the transmission of two upper-layer IPX protocols, SAP (Service Advertising Protocol) and RIP (Route Information Protocol). The transmission and broadcast of SAP and RIP information are necessary for the normal operation of NetWare client-server interactions. A typical client startup sequence demonstrates the use of SAP and IPX RIP:
When a DOS NetWare client 'attaches' to the network, i.e. loads the NetWare Shell (NETX.exe) or DOS Requester (VLM.exe), it issues a Nearest Service Query with service type equal 04 for file server. All IPX routers, including NetWare servers, on the subnet respond with SAP information of type 04, in the form of a Nearest Service Response. The client uses this information to build its SAP table. The client responds to the first SAP response it receives by broadcasting an IPX RIP request for that server's internal IPX network. Either the server or the router that originally responded will issue a RIP response with the path to the server. The client then issues a Create Service Connection request to the file server process. Once connected the client will ask this server for the internal network address of its preferred server, if configured. The nearest server responds with that information and the client starts again by broadcasting an IPX RIP request for the preferred servers' internal network number. Once the client is attached to a server a user can login to it or any other server.
Servers send General Service Responses (broadcasts) and RIP broadcasts once every minute. IPX RIP implementation includes split horizon and poison reverse. (Split horizon is also followed for SAP broadcasts.) SAP and RIP traffic on the backbone would be approximately two SAP packets and one RIP packet from every router once per minute, or approximately .04 Mb/s. On a subnet it would be ten to twenty SAP packets and one RIP packet per second, plus local server broadcasts, or .007 Mb/s.
The Proteon IPX router module contains three mechanisms for controlling IPX RIP and SAP information. They are access controls on IPX sockets, SAP filters and a toggle for replying to Get Nearest Service requests. Access controls are entered for the router not each interface. They control whether packets are processed and forwarded or dropped by the router. Access controls are either of inclusive or exclusive type, specify valid source network, node, and socket range and destination network, node and socket range.
Routers (not interfaces) can also be configured to forward only certain kinds of SAP packets termed SAP filtering. SAP filters are based on service type and hop count, e.g., advertisements of print services could be limited to local subnets. While SAP filters could be used to limit the amount of SAP traffic on the backbone they were typically used to protect older NetWare servers that had limited SAP table space.
Routers can be configured not to respond to Get Nearest Service queries, thereby keeping a client's first 'attachment' to a local server, if present.
4.1.2. Misconfiguration Issues
There are two kinds of typical IPX network misconfigurations: two routers claiming different network numbers for the same external network and two routers claiming the same network number for different networks, either internal or external. These misconfigurations are germane because NetWare servers, with their internal networks, are routers and they are configured by departmental system administrators.
When two routers claim different network numbers for the same external network both will immediately report the error. NetWare server consoles will display the error message, "Router at XX-XX-XX-XX-XX-XX claims network NNNNNNNN should be MMMMMMMM." Proteon routers' Event Logging System can be configured to report IPX.052 (RIP response from wrong network) and IPX.053 (SAP response from wrong network) errors to the monitor process.
While this error condition exists, a NetWare client will believe the first router from which it receives a RIP reply. The client will use the external network of the 'first' router. It will not be able to reach desired services if they are not available through the 'first' router. At best, connection to services will be inconsistent.
When two routers use the same network number for different networks, a more complex error condition exists that is not as easily detected. Consider these two misconfigurations: the first is the case where a server is configured with an internal network number that belongs to an external network elsewhere in the internetwork.
Server 1 (S1) is incorrectly configured with internal network
number 802022. This is the network number of the subnet connected
via Router (R2). IPX RIP entries consist of the IPX network number,
time to reach the network in ticks (1/18th of a second), hop count,
and the network address of the intermediate router. Routers determine
the best route to a network by evaluating tick count or, if two
routes with equal tick counts exist, by hop count. R1's route
table entry for network 802022 will show that it is n ticks
away through network 802011 and S1. R2 is directly connected to
network 802022; its route table entry to this network is essentially
statically configured. R3 will hear two RIP broadcasts for network
802022, that it is n + x ticks and 2 hops away via
R1, and x ticks and 1 hop away through R2. It will use
the second route because of the lower tick count. Clients on network
802022 and 802033 will be unable to reach the internal network
of S1 and the server will be effectively unreachable. Clients
on network 802011 will be able to reach S1's internal network
and use that server, but they will be unable to reach S2.
This state could be difficult to detect, particularly if there is little need for internetwork communication to or from network 802011 and S1. It would, however, be fairly straightforward to detect the source of the misconfiguration once discovered.
The second case is where a server is configured with an internal network number belonging to another server:
S1 is configured with an internal network number that belongs
to S2. R1's route table entry for 80202222 shows it is mticks
and one hop away via S1. R2's route table entry shows it is n
ticks and one hop away via S2. R3 will receive RIP broadcasts
that network 80202222 is m + x ticks and 2 hops
away via R1, and n + x ticks and 2 hops away via
R2. It will route to the network with the lowest tick count. If
the tick count changes over time, R3 will flip between R1 and
R2 as the intermediate router for network 80202222. This has the
effect of breaking any existing connections from clients on 802033
to either S1 or S2. Clients on 802033 will have inconsistent,
and failure-prone connections to S1 and S2. Clients on 802011
will be unable to reach S2. Clients on 802022 will be unable to
reach S1.
This state, is likely to be detected earlier and is potentially more destructive. R3 represents all other routers on the backbone ring. All clients, (except those on 802011 and 802022 which have no connection to S2 and S1, respectively), in the IPX internetwork would have inconsistent and failure prone connections to S1 and S2.
The existing "ring of trees" campus network design reduces the kind and complexity of potential misconfigurations and makes it easier to troubleshoot misconfigurations. (The current IPX network numbering scheme should make it easier to spot these types of misconfigurations by simply reading a campus router's IPX RIP table.) In non-standard configurations, such as where a server is more than one router away from the ring, more complex problems could occur. In addition, it is not necessary to consider a subnet being misconfigured with the network number of another subnet. This assumes that campus routers will be correctly configured for directly connected subnets and only the internal networks of servers might be misconfigured in these ways.
If an unregistered or misconfigured NetWare server, or an illegal IPtunnel was discovered, exclusive access controls could be used to protect the campus network from the misconfigured server and whatever is beyond it.
4.1.3. Cost
The current Proteon CNX500 IPX module was free. The IPX and AppleTalk modules will be included in all future releases at no extra cost. It is recommended that memory be added to the CNX500 routers and some P4200 routers be replaced. The cost of upgrading every router to a total of 2 MB of memory would be approximately $30,000. The cost of replacing or upgrading the connections from departments currently connected via is P4200 routers is $56,000.
4.1.4. Support
Network Operations and Development (NOD) would be responsible for configuring the IPX module in campus routers. This involves enabling IPX globally, configuring the IPX address and Ethernet frame type for each interface, and entering access controls if used. In addition to the current error conditions reported by the router, several from the IPX group should also be added to the monitor subsystem. When troubleshooting a network problem, NOD must be able to verify the correct transmission of IPX packets, as well as IP. The LANalyzer is fully capable of decoding IPX packets.
Network Services (NS) would maintain IPX address space and troubleshoot misconfigured NetWare servers. NS would also be responsible for evaluating IPX support in routers and other network devices. Other than correctly configuring their NetWare services, a necessity in any case, departmental system administrators would have little or nothing to do with the maintenance of the IPX internetwork.
4.1.5. Client, Server and Router Issues
NetWare servers are not the only devices that advertise services via SAP. Many products are available on the market that provide special services to NetWare clients. The two most common types of third party servers used on this campus are print servers and CD-ROM servers. These products typically use one of two designs. The first is a client DOS computer dedicated to providing the service. The second is a special purpose device configured with enough logic, usually in ROM chips, to boot, advertise its service and perform it. The most common example of this type of device is a print 'server': a small box with a network connection and a printer port. Through these devices, printers are directly connected to the network. Third party servers would function as designed in an all IPX environment. They would advertise services via SAP and that information would be broadcast throughout the IPX internetwork. These servers could be placed where they are needed independent of the location of NetWare servers.
If IPX were routed each router would have to load the IPX module and maintain SAP and IPX RIP tables. The memory requirement for IPX support in the routers is approximately 100 KB.
Older Proteon P4200 routers do not support IPX routing. Departments which are connected to the campus network by these routers would not be able to participate in an IPX internetwork. They would have to rely on IPtunnels or NetWare/IP. Currently this would affect a dozen departments and twice as many NetWare servers.
4.1.6. Experiences
We tested IPX routing on a non-production router for several months without incident. We are currently testing IPX routing on two production routers, including their FDDI interfaces, without problem or interference with IP routing. These tests have taken place within the existing IPX internetwork which includes IPtunnels and NetWare/IP servers. The current IPX internetwork includes over 100 server/services and over 50 IPX networks.
4.2. Using NetWare/IP
4.2.1. Description
NetWare/IP software allows NetWare servers and clients to communicate via IP network communication rather than IPX. A NetWare/IP implementation would not involve any configuration changes in campus routers.
NetWare/IP is a collection of NLMs that replaces certain functions of the IPX protocol so that upper layer IPX protocols continue to function without IPX. "The IPX protocol has broadcast mechanisms that propagate network-wide information. Many NetWare applications ... rely on this information. Because IP has no counterparts for the IPX internetwork broadcast capabilities, NetWare/IP provides two services that emulate these mechanisms and eliminate these broadcasts on IP segments." These services are the Domain SAP/RIP Service (DSS) and the Domain Name System (DNS). The DSS server maintains tables of two kinds of information necessary for NetWare services and servers: SAP and IPX RIP. SAP is used to broadcast the services provided by specific NetWare servers in the internetwork and IPX RIP is used to broadcast the routes to NetWare services.
NetWare servers can be configured to use both IP and IPX protocols. NetWare DOS clients can use only one protocol to communicate with a NetWare server. There are several network design possibilities.
One option is to configure all NetWare servers and DOS clients to communicate via NetWare/IP exclusively.
Alternatively, DOS clients could continue to communicate with
NetWare servers on their local subnet via IPX. At least one NetWare
server on each subnet would need to be configured with the NetWare/IP
protocol stack in addition to IPX; these servers are termed Gateway
servers.
When a client needed to communicate with a server on
another subnet, the local server would transfer IPX-based communication
to NetWare/IP for transport across the backbone. NetWare clients
located on a subnet without a NetWare server would be configured
as NetWare/IP clients.
NetWare IPX servers receive information about the internetwork through IPX RIP and SAP broadcasts. NetWare/IP servers register and learn SAP and IPX RIP information from the nearest DSS, via UDP (User Datagram Protocol). SAP information in the DSS database expires so NetWare/IP servers re-register at a configurable interval from 1 to 60 minutes, (default 5).
There is only one primary DSS in a NetWare/IP domain. To improve performance and reliability at least one NetWare/IP server on each subnet should be configured as a secondary DSS. The primary DSS holds the SAP and IPX RIP database. Read/write replicas are kept on secondary DSSs. SAP information is registered at all DSSs and their databases can become out of synchronization. Secondary DSSs request synchronization from the primary DSS at regular and configurable intervals (1 to 1000 seconds, default 60), if connectivity between secondary and primary DSSs is lost and then restored, or when a secondary DSS boots. Only changes, additions and deletions to the SAP/RIP database are transmitted between DSSs. In contrast, NetWare IPX servers broadcast their entire SAP and IPX RIP tables every minute, and this is not configurable.
At startup, a NetWare/IP client sends a DNS lookup to a campus DNS server asking for the location of the DSSs for its NetWare/IP domain; it does this by requesting the IP addresses for the name-servers for the NetWare/IP domain. The client also issues a UDP broadcast on the local subnet looking for NetWare/IP servers. If none respond it sends the NetWare/IP equivalent of a Get Nearest Service request to the first DSS for SAP information. The DSS responds with the names and addresses of up to five NetWare/IP servers. The client could then send a Service Connection Request to the nearest or preferred server.
A white paper from Novell explains how NetWare client applications are able to function on top of a TCP/IP stack:
IPXODI.com is responsible for end-to-end transport of data between systems. It exports an Application Binary Interface (ABI) called the IPX Far Call Interface... The NetWare libraries, shell and applications ... use the IPX layer to transmit data by making function calls into IPXODI.com using the IPX Far Call Interface. NWIP.exe [exports] the IPX Far Call Interface to applications, libraries and shells above it while making the appropriate calls to TCPIP.exe below it.
NWIP.exe successfully fools all upper-layer applications tested to date. Presumably any network application that follows the IPX and ODI specifications will work with NetWare/IP.
In a mixed IPX and NetWare/IP network design IPX clients would operate normally with a nearby NetWare IPX or gateway server. Gateway servers would use the mechanisms of NetWare/IP when communicating off-subnet, and DSS registration to participate in the NetWare internetwork. They would use the mechanisms of IPX SAP and RIP broadcasts to communicate with clients and servers on the local subnet.
4.2.2. Misconfiguration Issues
It is not clear what the primary DSS does with conflicting route information. It should believe the secondary DSS with the best route to a network and may alternate between ambiguous best routes. Secondary DSSs should believe the primary DSS. Every time a misconfigured NetWare server registered with its nearest DSS it could trigger a DSS-synchronization storm.
A NetWare/IP server, at some other non-campus location or part of another domain, could be configured to register with our DSS simply by knowing the name of our NetWare/IP domain. If that server was participating in an IPX internetwork it would register those IPX networks with our DSS.
4.2.3. Cost
The current cost of NetWare/IP for a campus site license would be $1000 per NetWare/IP server or $70,000.
4.2.4. Support
NOD is responsible for troubleshooting problems with the campus network infrastructure primarily in the physical, datalink and network layers. When a problem is reported they will verify that all network devices are functioning properly, that router interfaces are addressed properly for IP RIP, and that IP packets are successfully transmitted. NetWare/IP does not change NOD responsibilities or workload.
NS is responsible for troubleshooting network problems in upper protocol layers. NS would therefore be responsible for adding DSS NetWare/IP servers to the campus DNS database, for operating the primary DSS and supporting the NetWare/IP internetwork. WSSG and NS would assist departmental system administrators with the installation and configuration of NetWare/IP servers. NS would be responsible for troubleshooting configuration errors, communication problems between clients and servers, and maintaining the IPX address space. If necessary, NS would assist departments in the design of an IPX-NetWare/IP network.
4.2.5. Client, Server and Router Issues
NetWare/IP servers would need an additional 2 MB of memory to load the necessary NLMs including DSS.NLM.
Third party servers would work on a NetWare/IP stack if it is possible to configure the network drivers they use. Servers that are dedicated clients using ODI network drivers could load TCPIP.exe and NWIP.exe instead of IPXODI.exe. If the server software is completely compatible with the ODI specification then it should work. Other servers, such as print server devices, where the device's network drivers are compiled into a boot image or written on ROM chips, would not work without a nearby gateway server.
Manufacturers of several third party servers were contacted for information regarding their compatibility with NetWare/IP; several used on campus have been tested. So far, two dedicated client print servers and one dedicated client CD server would work in a NetWare/IP environment.
The Novell TCP/IP stack used by NetWare/IP is compatible with the TCP/IP applications in Novell's LAN WorkPlace for DOS. Departments that wish to participate in a campus-wide IPX internetwork and use other TCP/IP applications such as those by FTP Software or Clarkson University, would be forced to convert to LAN WorkPlace if NetWare/IP was implemented. They could continue to use their chosen TCP/IP stack if IPX was routed. In the future most TCP/IP applications for MS Windows will be based on winsock.dll and will be independent of the particular TCP/IP stack loaded.
It is currently possible to load ODIpkt.exe, a packet driver shim, that allows TCP/IP applications, that normally require packet drivers ,to run on Novell's TCP/IP stack. There is, of course, memory overhead in running these driver shims. FTP Software recently announced plans to design its TCP/IP applications to run on Novell's TCP/IP protocol stack.
Campus routers would not be affected, other than by the presumed increase in traffic among NetWare/IP servers. DCNS would be responsible for the administration of the primary DSS, and departmental system administrators for the administration of NetWare/IP clients and servers.
4.2.6. Experiences
There are currently four NetWare/IP servers on campus, including the primary DSS and a NetWare 4.01 test server. The NetWare 4.01 server is not using IPX at all; all NetWare Directory Services function normally. Two servers, including the primary DSS, are also production servers. Other than minor installation glitches, there have been no problems in the operation of these servers. Novell's testing suggests performance hits of 4-8%.
NetWare/IP traffic cannot be estimated at this time but includes NetWare/IP servers registering with their nearest DSS every five minutes, DSSs synchronizing their databases every 60 seconds, and DSSs refreshing DNS information with campus name servers. The traffic could be more than IPX SAP and RIP broadcasts.
NetWare/IP could be implemented as a total solution, i.e., all clients and servers would rely solely on NetWare/IP for network and transport. This configuration was tested in Design 2. Both on the same Ethernet and through a campus router, this design should be compared with an all-IPX configuration as in Design 1. (General-use servers, such as those used for distribution of site licensed software, could configure guest accounts with physical station restrictions limited to the UC Berkeley IPX external networks.)
The results of this comparison show that NetWare/IP on the same
Ethernet has 96% throughput of IPX. Through the router, NetWare/IP
has 91% throughput of IPX. While measurable, these differences
are quite tolerable.
All IPX Design vs. All NetWare/IP Design
KB/s Design 1: All IPX Design 2: All % of Design 1
NetWare/IP
Same Ethernet 1031.57 996.30 97
Through Router 969.21 885.76 91
Alternatively, NetWare/IP could be used as a replacement for current IPtunnels, i.e., as a way of translating IPX-based communication to IP for transport across the campus backbone. Design 3 is the configuration of an IPX client communicating with a NetWare/IP server through a gateway server. This design would be appropriate for situations where the client needs only occasional access to the NetWare/IP server. The case where the NetWare/IP server is not located on the same Ethernet would be the most likely scenario, although the same Ethernet configuration is shown for completeness.
Design 4 is the configuration of a remote client with no nearby
gateway server. It is therefore a NetWare/IP client and accesses
an IPX-based server via a gateway server. Again, the same Ethernet
configuration is shown for completeness.
All IPX Design vs. Gateway Designs
KB/s Design 1: All Design 3: IPX Design 4: % of Design 1
IPX client, NetWare/IP NetWare/IP
server client, IPX
server
Same 1031.57 525.00 524.38 51
Ethernet
Through 969.21 511.84 511.17 53
Router
All NetWare/IP Design vs. Gateway Designs
KB/s Design 2: All Design 3: IPX Design 4: % of Design 2
NetWare/IP client, NetWare/IP NetWare/IP
server client, IPX
server
Same 996.30 525.00 524.38 53
Ethernet
Through 885.76 511.84 511.17 57
Router
These designs should be compared with both the all-IPX and all-NetWare/IP designs. Through a campus router, both gateway configurations are 53% throughput of the all-IPX design, and 58% of the all-NetWare/IP design. For those cases where remote access is needed only occasionally, forcing this traffic through a second server is a design option, albeit a costly one. Remote clients should simply be configured as NetWare/IP clients. The cost of supporting both protocol stacks in a server is acceptable, as long as the clients rarely use the server to translate between IPX and NetWare/IP. Departments with a significant and consistent need to access NetWare services across the backbone should consider converting completely to NetWare/IP. The Library, for example, will have many clients accessing many servers, both local and remote, at all times. The NetWare/IP gateway model would be inappropriate in this situation.
For the NetWare users on campus it is assumed that routing IPX would meet their needs. Therefore only the appropriateness of NetWare/IP is examined in the next section. (One glaring exception are the departments connected to the campus via P4200 routers. As those routers do not support IPX routing their only solutions are IPtunnels or NetWare/IP.)
6.1. Departments with Multiple Subnets
Large departments with multiple subnets need access from all clients to all servers; they need the flexibility to place services on any subnet; and they need sufficient throughput to launch applications across the campus network. A NetWare/IP-IPX Gateway design would not be sufficient for either the Library or the College of Chemistry. These departments would need to convert entirely to NetWare/IP. Unfortunately most third party servers are not currently supporting NetWare/IP and may not for many months. Until this support is available these departments would be forced into network design configurations as inflexible as the current IPtunnel solution. Workstation and Microcomputer Facilities could implement a NetWare/IP-IPX Gateway design now, and convert to all NetWare/IP as they reduce the number of NetWare servers they support. NetWare/IP does not meet the needs of large, multi-subnet departments at this time but may in the near future.
6.2. Remote clients
Departments with a small number of NetWare clients located on different subnets from their NetWare servers could easily implement NetWare/IP. Even though the NetWare servers would be supporting both protocols very little client communication would be translated by the NetWare/IP Gateway.
6.3. Campus information service
These servers would only be available to NetWare/IP clients or IPX clients with nearby NetWare/IP gateway servers. For occasional access, users of these systems could create a NetWare/IP boot diskette. For more consistent access, the client or its nearby server would have to run NetWare/IP.
6.4. NetWare for Home
The campus annex terminal servers currently support only IP. Any dial-in NetWare service managed by DCNS would have to operate on a IP transport. (There are several IPX-based dial-in network access products currently available; individual departments may, in fact, be using them now.) It may also be possible in the future to load IPX support on the terminal servers. Given the high demand on and current reliability of these machines now, this must be seriously considered. In addition, the campus modems are currently behind P4200 routers.
A beta-release version of LAN WorkPlace for DOS, termed Mobile WorkPlace, is optimized for SLIP/PPP connectivity. LAN WorkPlace TCP/IP applications work well over this SLIP/PPP network driver. However, the NetWare client currently included is IPtunnel.exe, not NWIP.exe. NetWare/IP was tested in this SLIP configuration and did not work. Current connection to NetWare servers via SLIP/PPP is highly unreliable.
There are several relevant issues in evaluating the two solutions for integrating NetWare services with the Berkeley campus network. These issues range from the pragmatic to the philosophical. They are: support, cost, performance, the costs of heterogeneity, internetwork context, and the use of proprietary protocols vs. promoting non-proprietary protocols.
7.1. Support
If implemented, NetWare/IP would be installed and administered by departmental NetWare system administrators, presumably with consulting support from WSSG and DCNS (Network Services). DCNS would also need to maintain the primary DSS, a relatively minor but ongoing support issue. Routing IPX, on the other hand, significantly adds to the configuration effort for every existing and new router. In addition, DCNS (Network Operations and Development) staff must be able to troubleshoot the IPX protocols. They will need to understand the protocols, how they work and how they break, and how to use protocol analyzers with the IPX protocols.
From the standpoint of troubleshooting a broken NetWare service, routing IPX is a more straightforward model to support than NetWare/IP. IPX clients server interactions would function as designed, without emulation and via native protocols.
7.2. Cost
There is a significant difference in the cost of Proteon's IPX module and NetWare/IP; the IPX module will be available at no extra charge and NetWare/IP costs $1000 per server, for all current and future servers. Unfortunately there is also conflicting information about the price of NetWare/IP from Novell. Until resolved, purchasing NetWare/IP for even some of the servers on campus would be expensive. While the Proteon IPX module is free, it would be prudent to add memory to the CNX500 routers and replace older routers. This adds to the cost of routing IPX.
Also at issue is where the cost would be borne. All departments wishing to participate in the campus IPX (or NetWare) internetwork would share the cost or IST could subsidize the purchase.
7.3. Performance
While there was a minor performance degradation with NetWare/IP the difference is clearly tolerable. And it is important to note that when a better designed LAN adapter, one optimized for parallel tasking, was used in the NetWare gateway server, the throughput of all gateway designs improved to 700-750 KB/s.
7.4. Client, Server Router Issues
A slight advantage in using IPX clients relates to the memory usage in DOS/Windows clients by the network drivers. An IPX client has a memory overhead of approximately 90 KB (loading LSL.com, an MLID driver, IPXODI.com and VLM.exe.) If this client also loaded TCPIP.exe in order to use TCPIP applications such as telnet and FTP, the memory usage for network drivers would be 115 KB. A client loading the NetWare/IP configuration (LSL.com, an MLID driver, TCPIP.exe, NWIP.exe and VLM.exe) uses approximately 120 KB.
7.5. Costs of heterogeneity
Heterogeneity introduces additional complexity in all aspects of managing the network:
Every protocol family consumes resources including actual overhead packets that consume bandwidth, share-of-mind, training, network management and router resources. Mean-time-between-failure is often a function of complexity of the network environment; the fewer protocols you have, the less likely they are going to interact with each other in unpleasant ways. Mean-time-to-repair is often a function of complexity of the network environment; the fewer protocols you have to think about in solving a network problem, the faster you will be able to do it. Support for multiple protocols can make the majority unwitting victims of the few. Example: a protocol-specific memory leak in a router could take everyone out of service, even if the bug was in a protocol that only a few people used.
Simply put, "In order to survive and provide the highest quality support, the organization must identify a minimal set of key technologies. It must decide not only which technologies it will actively support, but which ones -- while not centrally supported -- are innocuous, and which ones should be actively discouraged because they will ultimate in higher support costs for the university."
When selecting new networking hardware, e.g. ATM switches, the necessity of supporting two protocols increases the complexity of the selection process, decreases the available options, may postpone the introduction of new technology to the campus network (because devices that support both protocols are not yet available), and adds to purchase price.
7.6. Internetwork Context
While it is clear that departments require the integration of NetWare services with the campus backbone, the UC Berkeley campus network does not exist in isolation. It is a significant part of BARRNet (Bay Area Regional Research network), our access to the Internet. Historically, UC Berkeley, BARRNet, UC Office of the President and other UC campus have implicitly agreed to use only the TCP/IP protocol suite. Any solution that DCNS chooses to implement may have political repercussions with these organizations.
7.6. Proprietary vs. Non-proprietary Protocols and Philosophy
An enormous strength of the Internet is its reliance on standards and protocols developed in a public forum. The process resulted in better standards and created a market which demanded that vendors and developers of network products comply with a suite of well-integrated and robust protocols. Network administrators could connect heterogeneous systems on a consistent and well-known network foundation. When commercial developers embark on a product development venture that relies on non-standard, or proprietary protocols, many believe they should be made to feel the market pressure of the Internet.
The main reasons given by institutions for not routing IPX include a desire to influence vendors of network products to use only non-proprietary standards (by refusing to use their products if based on proprietary protocols) and a desire to limit heterogeneity. These institutions typically have very large networks and are considered leaders in the Internet community.
The institutions that are routing IPX typically have smaller networks and a greater concentration of Macintosh and DOS NetWare clients. The reasons they give for routing IPX are user demand and ease of administration. Two notable exceptions having large networks are the State of Utah and the University of Michigan:
The educational institutions, high and low, of the State of Utah route IPX to all places. That means we have a huge state IPX network using ciscos. Careful filtering keeps things compact and protected at each site. We have several thousand registered IP addresses (Class B site), about a hundred NW 3/4 servers, uncounted Ks of PCs, a countable set of Macs, hundreds of Unix machines, and VAXen. NetWare is very widely used here. Client population is pushing 20K people on about 1/5 that many machines.
Many of (the departments) have selected to use NetWare without the input of the central IT office. U of Michigan has 200+ NetWare servers and we want to provide those departments with good service. So we support NetWare and we route IPX on our FDDI backbones. We've been routing it for five years. It's reliable and takes very little management. It has also been the easiest thing for our departmental administrators to set up.
It is important to note that of the institutions that route IPX all of them use Cisco routers, particularly Cisco AGS+, and the Berkeley network uses Proteon routers. Some also use NetWare Multi-Protocol Routers and NetWare servers as routers.
The issues, advantages and disadvantages are summarized in the following table:
ISSUE NetWare/IP Route IPX
Client memory usage 120 KB 115 KB (including TCP/IP
(approx.) stack)
Server memory usage 2 MB 0
(approx.)
Router memory usage 0 100 KB
Cost (changing $70,000 (or more) $86,000
constantly)
Paid by Could be shared among IST capital improvement
departments using NetWare or funds or DCNS development
paid by IST. funds.
Heterogeneity No Yes
Performance All NetWare/IP design has
90% throughput of all IPX
design.
Gateway IPX/IP designs are
55% of all IPX design.
Administration By departmental system All DCNS. NOD would
administrators, with WSSG configure and
and DCNS consultation. NS trouble-shoot router
would run DSS server. configuration.
Training Needs No Yes. Significant
training needs for NOD.
Promotes non-proprietary Yes No
standards.
Novell will continue to Yes Yes
support protocol.
Dial-in or Home NetWare Yes. Current beta-version Not soon or ever. Would
of Mobile LAN WorkPlace require IPX support in
uses IPtunnel.exe client terminal servers.
rather than NWIP.exe client.
TCP/IP stack Requires use of Novell's Allows departments to
TCPIP.exe. continue to use
non-Novell TCP/IP stack.
Other IPX module not available
for older P4200 routers.
8. Conclusions & Recommendations
Neither solution is ideal. NetWare/IP is imperfect because it currently costs too much and it will not integrate 100% of current services and clients. However, it does not add heterogeneity to the campus network infrastructure and it promotes non-proprietary standards and solutions. The current problems with NetWare/IP are likely to improve with time, particularly if the network market demands it from both Novell and vendors of third-party server products.
Routing IPX is the only solution that will meet the needs of the campus NetWare users now. It is a simpler model for providing connectivity between NetWare services and clients. It will however put a significant demand of time, energy and resources on Network Operations and Development. The demand on Network Services will be similar no matter which solution is implemented; it is simply the demand of integrating NetWare services with the campus network. Routing IPX on the campus network will impact all aspects of running, troubleshooting and upgrading the network; it also sets a precedent.
There are two choices. The first is an imperfect solution, albeit likely to improve over time, that maintains the non-proprietary, single-protocol integrity of the backbone. The second is a solution that meets the needs of a significant number of campus network users with a more straightforward model for those users and burdens DCNS with significant support costs.
After careful evaluation, we have determined that while promoting non-proprietary solutions is important, given the significant need of NetWare users for a technically viable solution available today, the appropriate choice is to route IPX and absorb the support costs.