Thursday, December 25, 2008
Lambi Judai - Jannat
Char dino ka pyaar o rabba
E-------------F#
Lambi judai lambi judai ...........Heee rabbaaaaaaaaaaaaaaaa
G#m------------- F#
Tere bin dil mera lage kahin na
G#m------------- F#
Tere bin jaan meri jaye kahin na
G#m-------------F#
Kitne zamane baad o rabba
E---------------F#
Yaad tu aaya, yaad tu aaya
Aaaaa.....aaaaaaaaaaa.........
G#m---------------F#
Khoya raha mein, saazon mein apne
E----------------F#
Aahat bhi teri, bhool gaya hoon
G#m-------------F#
Kitna jeeya hoon tanha raho hoon
E----------------F#
Ishq tera bhool gaya hoon ------stop-----------
G#m-------------F#
Uljha raha mein is zindagi mein
E--------------------F#
Dil ki duhai dil ki duhai
Am
Ho rabbaaaaaa..rabbaaaaaaaah......kitheeee mai jaawaaa…
C
haaaaallllllllll sunaawaaaaaa,
G C Am E
ve dasso rabaa….aaaaaa….aaaaaaaaaaaaaaaaaa
G#m---------------F#
Har bebasi mein, is zindagi ne
E-------------------F#
Tujh ko hi chaha, tujh ko hi manga
G#m---------------F#
Jin raston se guzra yeh dil tha
E-------------------F#
Manzil mili na pyaar na paya -------stop--------
G#m---------------F#
Khud ko chupake rahon se guzre
E-------------------F#
Dil ko sambhle dil ko sambhleeeeeeeeeeeeeeeeeeeeee
Tere bin dil mera lage kahin na--------------------
Friday, December 12, 2008
Beete Lamhe - The Train
dard mein bhi yeh lab muskura jaate hain
------C----------------------D6---------D
beete lamhein hamein jab bhi yaad aate hain...
-----Em---------------D6----------D--
dard mein bhi yeh lab muskura jaate hain
------C----------------------D6----------D
beete lamhein hamein jab bhi yaad aate hain...
------Em---------Em/D(or Dsus4)--C----Em/D(or Dsus4)
beete lamhein --------------------------------------
G--------------Bm-----C-------D---
Chand lambhaat ke vaaste hi sahi
G--------------Bm-----C-------D---
Mushkura kar mili thi mujhe zindagi
G--------------Bm-----C-------D---
Chand lambhaat ke vaaste hi sahi
G--------------Bm-----C-------D---
Mushkura kar mili thi mujhe zindagi
-----C------D------------Bm-----Em
Teri aagosh mein din the mere kate
-----C-------D--------Bm---------Em--------D-----
Teri baaahon mein thi mere raatien kati....ii..i..
---C----D----
Wo ou ou oooo
----Em---------------------D6
Aaj bhi jab woh pal mujhko yaad aate hain
-------C-------------D6-------D
Dil se saare gumo ko bhoola jate hain
Em-----------------------D6--------D
Dard mein bhi yeh lab muskura jaate hain
C----------------------------D6---------D
Beete lamhein humein jab bhi yaad aate hain
Em-------D6-------Cmaj7-----D----
Beete lamheinnnnn ...
Am-D-
-----Em------------D------Cmaj7----D
Mere kandhein mein sar ko Jukana Tera
-----Em--------D-------Cmaj7------D
Mere sene mein khud ko chupana teraa
-----Em------------D------Cmaj7----D
Mere kandhein mein sar ko Jukana Tera
-----Em--------D-------Cmaj7-----D
Mere sene mein khud ko chupana teraa
-----C------D---------Bm--------Em
Aake mere panaho mein shaam-o-seher
---C-------D-------Bm-----Em------D---
Kanch ki trah woh tut jana tera..aa..a..
C----D----
Wo-----
----Em-------------------D6-----D
Aaj bhi jab woh manzar nazar aate hai
-------C--------------D6---D
Dil ki viraniyon ko mita jate hai
-----Em---------------D6-----------D--
dard mein bhi yeh lab muskura jaate hain
------C----------------------D6---------D
beete lamhein hamein jab bhi yaad aate hain...
Monday, December 1, 2008
mumma - dus vidaniya
Maa, Meri Ma, Pyaari Ma Mumma
F..................Bb........F
Hoh, Maa, Meri Ma, Pyaari Ma Mumma
F.....................Bb
Haathon ki lakeerein badal jaayengi
F.....................Bb
Gum ki yeh zanjeerien pighal jaayengi
F...........C......
Ho khuda pe bhi asar
F...........C......
Tu duaao ka hai Ghar
F..............Bb........F
Maa, Meri Ma, Pyaari Ma Mumma
F.................Bb
Bigdi kismat bhi savar jaayengi
F......................Bb
zindagi taraane khushi ke jaayengi
F...........C......
Tere hote kiska dar
F...........C......
Tu duaao ka hai ghar
F.................Bb
Yu to mein sabse nyaaraa hoon
F.................Bb
Tera maa, main dulaaraa hoon
F.................Bb
Yu to mein sabse nyaaraa hoon
F...................Bb
Par tera maa, main dulaaraa hoon
F...........................Bb
Duniya mein jeene se zyaada uljhan hai maa
F...............C..........F....
Tu hai amar ka jahaaaan aaaaaaaaa
F........................Bb
Tu gussa karti hai bada achha lagta hai
F.........................Bb...................C.......
Tu kaan pakadti hai, badi zor se lagta hai, meri maaaaa
F.............Bb........F
Maa Meri Ma Pyaari Ma Mumma
F.....................Bb
Haathon ki lakeerein badal jaayengi
F.....................Bb
Gum ki yeh zanjeerien pighal jaayengi
F...........C......
Ho khuda pe bhi asar
F...........C......
Tu duaao ka hai Ghar
F..............Bb........F
Maa, Meri Ma, Pyaari Ma Mumma
Tuesday, November 25, 2008
Key Facts - Cisco 640 802
~ Only the Hardware addresses change when packets go through routers.
~ Half duplex Ethernet - One station can only send or receive at any time.
~ Ethernet Frame - 64bytes Min 1518bytes Maximum.
~ ISL frames are 1522bytes long, this can be mistaken for Giants and lost. Have to use ISL NIC cards. On router interface use 'encapsulation isl 2' to use ISL frames on VLAN 2.
~ FX and SX are fibre media, 100VG-AnyLAN is twisted pair copper media.
~ Spanning Tree is IEEE 802.1d - created by DEC (Digital Equipment Corp).
~ BPDUs are Multicast frames, sent every 2 seconds. Blocked ports still receive BDPUs.
~ Forward delay - Time taken from listening to learning (approx 50 seconds)
~ Default IEEE bridge priority 32,768, used to select root bridge. If these are identical then switch with lowest MAC address is used.
~ ISDN Protocols - E = Telephone network standards, I = Concepts, Terminology, Q = Switching, Signalling methods.
~ ISDN Reference Points - R = non-ISDN device and TA, S/T = references point between NT1 and NT2, U = NT1 and ISDN network (US only)
~ TE1 = Device compatible with ISDN, TE2 = Device NOT compatible with ISDN, TA = Converts non ISDN signals to ISDN signals, NT1 = Converts 4 wires into 2 wire local loop, NT2 = Providers equipment (Switch, PBX)
~ BRI - 2 * B-channel 64kbps, 1 * D-channel 16kbps (D-channel - LAPD)
~ PRI (Europe, Aus) - 30 * B-channel 64kbps, 1 * D-channel 64kbps (20.48Mbps)
~ PRI (EUS, Japan) - 23 * B-channel 64kbps, 1 * D-channel 64kbps (1.544Mbps)
~ ISDN supports IP, IPX, Appletalk...
~ ISDN can use PPP, HDLC, LAPD, each B-channel needs a SPID
~ Use static routes for ISDN otherwise it will keep link open.
~ MAC address 48 bits (12 Hex), IPX address 80 bits.
Netware 3.11 (1983-) - ethernet_802.3/novell-ether (Cisco default on~ Ethernet networks), Netware 3.12 or later (1985-) - Ethernet_802.2/sap - includes LLC, Ethernet_II - arpa, Ethernet_SNAP - snap, Netware 4.11 - use sap, Netware 5 uses IP
~ Novell RIP - Metrics = ticks and hops (15 max), 60 sec updates (tick = 55ms / 1/18 sec)
~ Novell 4.11 > uses NLSP (Netware Link Service Protocol) Link State Routing
~ SAP - Updates 60 Secs - 4 = Netware file server, 7 = Print server, 24 = Remote bridge server
~ Ping Responses - U = unreachable, C = congestion, I = user interrupt,? = unknown packet type, & = lifetime exceeded
~ Trace Responses - N = Network unreachable, !H = Not forwarded due to ACL restriction, P = Protocol unreachable, U = Port could not be reached
~ Ethernet 5-4-3 rule = Between 2 nodes there can only be max 5 segments, 4 repeaters and only 3 segments must have users.
~ 80/20 rule - 80% of traffic should be local 20% across backbone
~ Class 1 repeater (translational) - delay 140 secs, number you can use 1
~ Class 2 repeater (transparent) - delay 92 secs, number you can use 2
~ CSMA/CD - Used on half duplex devices
~ Auto-negotiate on FastEthernet checks link speed and duplex of line.
~ Protocol field in IP header - TCP = 6, UDP = 17, ICMP = 1, IGRP = 9
~ Ports - 20 FTP data, 21 FTP program, 23 - telnet, 25 - SMTP, 69 - TFTP, 53 - DNS, 80 - HTTP
~ Loopback address - 127.0.0.1
~ ACL - Standard ACL as close to destination as possible, Extended ACL as close to source as possible
~ IP = 1-99, Ex IP = 100-199, AppleTalk = 600-699, IPX = 800-899, Ex IPX = 900-999, IPX SAP = 1000-1099
~ Remember that there is an explicit ACL of 'deny all' if no statements match.
~ Multiprotocol routing supports more than one routing protocol, allows a router to deliver packets from several routed protocols.
~ Core Layer - High speed switching - free from filtering or anything which will slow packets etc.
Distribution Layer - Packet manipulation, address area segregation,~ broadcast domains, VLANs, security (ROUTERS), WAN access, queuing, firewalls, multicast domains, ACLs
~ Access Layer - End users, ACL/filters, remote access, shared bandwidth (SWITCHES), segmentation, DDR
~ HSSI - 52Mbps max
~ ATM cell size - 53bytes
~ Cisco LMI - DLCI - 16-1007, ANSI LMI - DLCI 16-992 (DLCI = 10bits)
~ LMI is a special DLCI = 1023
~ LMI Multicasting reserved for 1019-1022
~ LMI extensions - Virtual circuit status, multicasting, global addressing, simple flow control
~ LMI types Cisco (default), ansi, q933a. From IOS 11.2 LMI is auto-sensed
~ Class A - 1-126
~ Class B - 128.1-191.255
~ Class C 192.0.1-
~ Class D - (1110 highest order bits) - remaining bits for multicasting
~ Class E - (1111 highest order bits) - Reserved for future use
~ RIP 1 (Classful), single subnet, periodic updates of full routing table, max hop count 15
~ RIP 2 (Classless addressing), triggered updates, full routing table updates
~ Directed Broadcast - All host bits set to 1 received by all hosts on local broadcast domain.
~ Local Broadcast (255.255.255.255) - All bits set to 1 received by all hosts on local and remote broadcast domains.
~ Synchronous serial links default to HDLC on Cisco routers
~ VIP cards - type slot/port adapter/interface (e.g e/1/0/2) (remember first interface is 0 not 1)
~ IGRP Metrics - Delay, Bandwidth
~ Default route - ip route 0.0.0.0 0.0.0.0 172.16.20.1 - need to use 'ip classless' (Classless is enabled by default on IOS 12.x) (Only work on stub networks)
~ ip default-network 172.16.1.0
~ CDP timer default 90 secs, holdtime 240 secs
~ Trunked links - FastEthernet or GigabitEthernet only
~ Frame Tagging - ISL = Adds it's own FCS, Cisco propriety (default), IEEE 802.1q
~ LAN Emulation (LANE) - Used for multiple VLANS over ATM
~ 802.10 - FDDI Frame Tagging
~ Hosts can only communicate between VLANS using Layer 3 devices
~ VTP Modes - Server (Default for Catalyst switches) Need at least one server in a VTP domain. All changes are advertised. Client - Sends and receives updates. To make a switch a server make it a client first then promote it once it's VTP database has received the latest revision. Transparent - Does not participate in VTP domain, but forwards VTP ads through trunked links. They keep their own database.
~ VTP adverts sent every 5 mins or when a change occurs, changes only kept by other switches if higher rev no than their current version.
~ VTP pruning - If a switch does not have any ports configured for VLAN 5 then it won't receive updates for it. Disabled by default. Enabled across entire domain if configured. VLAN 1 is not pruning eligible.
~ Config Reg - 00 Rom Monitor, 01 Boot Image from ROM, 02-F NVRAM, Bit 6 set to 1 to ignore NVRAM. Register is 16 Bits.
~ 1900 Switch Config - enable password level 1 - usermode, level 15 - enable password.
~ 1900 switch can have up to 64 VLANS. You cannot telnet from a switch but you can telnet into it.
Administrative Distances
Routing Protocol
Administrative Distance
Connected Intf 0
OSPF 110
Static route 1
RIP 120
EIGRP 90
UNKNOWN 255
IGRP 100
~ RIP - Updates 30 secs, Max Hops 15, Invalid 90 secs, Flush 240 secs, metrics hops, load balance 6 equal cost links
~ IGRP - Updates 90 secs, max hops 255 (default 100), invalid 3x90 secs, holddown 3x90+10 secs, flush 7x90 secs, metrics bandwidth, delay, load balance upto 6 unequal cost links.
~ When routers are converging no data is sent.
~ Frame Relay - 64 kbps - 1.544 Mbps, non-broadcast multi-access encapsulation (NBMA), dynamic bandwidth allocation, congestion control. Can use PVC and SVCs, PVC more common. Virtual circuit established before data sent. Encapsulation Cisco (Default), IETF (use when connecting non-cisco routers). Static routes are more stable than IARP.
~ Routers are DTE devices by default, DCE interfaces need a clock rate.
~ Telneting uses layers 1-4 so a good test of functionality. If you type a command the router doesn't know or type and IP address it will try to resolve the name and telnet.
~ Bandwidth command sets cost for serial links. This is only used by routing protocols so they can 'cost' paths. Default = 1.544kbps (T1) Command is in Kbps.
~ Clock rate command is in bps.
~ HDLC - Connection-orientated, operates at the datalink layer, small overhead, no way of distinguishing network protocols. Every vendors implementation is different, NO authentication, CISCO Default over serial lines.
~ LAPB - Connection-orientated, datalink layer protocol, HUGE overhead, uses windowing, used instead of HDLC for error prone links.
~ PPP - industry standard, used when connection between different vendors devices. NCP to identify network protocol, authentication, compatible with async + sync links, operates at physical + datalink layers only. PAP - insecure authentication, CHAP auth provides initial + periodic auth. PPP compression uses stacker and predictor methods. Error detection - PPP uses quality and magic number methods. Multilink - IOS 11.1 only, spreads the load over 2 parallel circuits (bundle).
~ Ethernet 0 is up, line protocol is down - keepalive or framing issue, check keepalives on both sides should match, check clocking on DCE, check encapsulation on both ends.
~ Ethernet 0 is down, protocol is down, - carrier detect is not present, other end maybe administratively shutdown or interface or cable problem.
~ Ethernet 0 is administratively shutdown - the 'no shutdown' command has not been issued on the interface.
~ Show interface serial 0 - shows bandwidth, MTU, keepalives.
~ MTU default = 1500bytes.
~ Bandwidth default = 1.544Kbps (T1)
~ Keepalives default = 10 seconds.
~ Use a cross over cable to connect devices of the same type (e.g router Ethernet intf to router Ethernet intf)
~Cross over cables swap pins 1 and 3 RD, and pins 2 and 6 TX
~ STP - 10-100Mbps - 100metres
~ ScTP - 10-100Mbps - 100metres
~ UTP - 10-100Mbps - 100metres
~ Coax - Coaxial - 500metres
~ Fiber - Single Mode upto 3000metres
~ Fiber - Multimode upto 2000metres
~ Connectionless protocols rely on application layer protocols for error handling and delivery.
~ EIGRP holds separate routing tables for IP,IPX,Appletalk, but only uses one protocol to distribute the updates.
~ CDP uses SNAP (Subnetwork Access Protocol) to enable neighbouring devices to exchange data.
~ IPX NLSP - link-state routing protocol intended to replace IPX RIP and SAP
~ NCP - Netware Core Protocol - Provides clients with access to server resources
~ IPX SAP - Sent every 60 seconds - includes all known services.
~ sap is Cisco default for Token Ring networks, SNAP is default for FDDI networks
~ VTP allows VLANs to be trunked over Ethernet, ATM, LANE or FDDI
~ Gigabit Ethernet using Multimode Fibre can run up to 260m
~ 100BaseFX up to 400m
~ VLAN Management Policy Server - Must be configured with all hosts' MAC addresses for dynamic allocation.
~ Standard ping - 5*100 byte ICMP echos, time out 2 seconds
~ DHCP uses UDP packets
~ Passive interface command stops interface sending routing updates, but still receives them.
~ 2 ways to configure VLAN membership, statically or dynamically through VLAN Management Policy Server.
~ ISL and Trunk protocol used to configure trunking on a switch.
~ Pre 10.3 IOS commands Config Net - copy config from tftp to DRAM Config Mem - copy NVRAM to DRAM
~ IP routing table [administrative distance/composite metric]
~ IPX routing table [ticks/hops]
Cisco e-books
http://career-assessments.blogspot.com/2008/01/cisco-certifications-aspirants.html
Sunday, November 23, 2008
Setting up 2511 Router as Access Server
Set Up Cisco 2511 Access Server
Follow original document @ www.more.net ; Direct link is added above.This document is offered for informational support only. No active technical support is available and document revisions will not be made after the date noted at the bottom of this page.
Purpose
To give network administrators an understanding how a modem pool works and to guide in setting up a dial-in modem pool using the Cisco 25xx Access and Communication Server. An access server is used to provide dial-up access to a local network and Internet Resources. This document is not intended to provide setup parameters for users accessing the server.
Overview of Modem <==> Access Server Communication
The end-to-end topology for a dialin connection looks like:
Cisco Access Server | => | RS232 | => | access server modem | => | PSTN (Public Switched Telephone Network) | => | client modem | => | RS232 | => | client PC |
The Cisco Access Server and client PC are called Data Terminal Equipment (DTE), and the access server and client modems are called Data Circuitterminating Equipment (DCE). The Cisco Access Server uses three pairs of wires to connect the DCE to the DTE. In each pair, one wire transmits and the other receives. These pairs are TX/RX, RTS/CTS, and DTR/DCD. Each pair requires specific configuration on the DTE and DCE.
1. TX/RX Data Transfer
Transmit: DTE >TX> DCE
Receive: DTE <RX<>
Transmit and receive speed is set on the modem using the TX/RX wire pair. Notice that the DCE transmits on RX and receives on TX. The speeds at which the two devices are communicating on the RS232 must be the same. If they are not, you will get a speed mismatch, where either garbage or nothing appears on the screen when dialing in to the modem.
To set the port speed on the access server use the speed xxxxxx line configuration command. A complete example of setting up an access server line follows.
2. R TS/CTS Hardware Flow Control
Request to Send: DTE >RTS> DCE
Clear to Send: DTE <CTS<>
This pair of wires indicates the ability of a device to receive data. For example, when the DCE has a full data buffer and can no longer accept data from the DTE for transmitting, it will lower the CTS signal. When the access server can no longer accept data, it lowers the RTS signal. Both the access server and the modem must agree to hardware handshake with CTS/RTS. If they do not agree on handshaking, they will tend to overflow each other's buffers. Dropped characters or packet errors are typical signs of a handshaking mismatch.
To set the access server async port to use CTS/RTS handshaking use the flowcontrol hardware command. An example of setting up an access server line folows.
Configuration commands vary from modem to modem. Look for "Hardware Handshaking" or "RTS/CTS Flow Control" in the modem manual.
3. DTR/DCD Modem Control Data Terminal Ready:
DTE >DTR> DCE
Data Carrier Detect: DTE <DCD<>
This pair of wires is used between the DTE and the DCE to initiate and receive calls. When the access server is ready, DTR output is high. The access server lowers DTR to drop any existing calls and return to the stored configuration. The modem uses DCD output to indicate that a call has arrived that needs servicing by the access server. The modem drops DCD to indicate loss of the call.
The access server and modem must agree on the function of DTR and DCD.
Prerequisites
Before setting up the Cisco 25xx Access Server you must have:
- A dedicated connection to the MOREnet backbone or plans for installation
- Assigned IP addresses
- A facility capable of supporting power requirements for the access server and the modems attached to it
- Knowledge of TCP/IP based applications, particularly telnet and a workstation for telnet access to the access server.
Equipment For Setting Up Modem Pool
These are step-by-step instructions for setting up the Cisco 2511 or 2512 Access Communications Server. Please be aware that your access server comes with full documentation of hardware and software setup configuration guides on Cisco's UniverCD CD-ROM.
Note: Completion of these steps may require some network downtime.
Unpack Your Access Communication Server
The following items are included with each device:
1 | Cisco 2511 or 2512 Access Communication Server |
1 | v.35 Serial Cable |
2 | Octal cables kits with RJ45 plugs and 16 DB25 to RJ45 converter hoods |
1 | Rack/Wall mount kit |
1 | Console port attachment kit 1 DB25 to RJ45 Terminal (DTE) connector 1 DB25 to RJ45 Modem (DCE) connector 1 "rolled" UTP flat ribbon cable with RJ45 on each end |
1 | Power Cord |
1 | AUI-LAN Transceiver |
1 | UniverCD CD-ROM |
Unpack Your Modem 1 USR, MultiTech or SupraFax Modem | |
1 | Power adapter |
Hardware Setup
-
Unpack the contents and check to be sure all equipment is included. If anything is missing please contact MOREnet Network Operation Cener.
-
Attach the Octal Cables to the back of the access server. In most cases it is easier to attach these cables first. If your particular physical environment makes this impractical, simply complete each instruction in the best order for your situation.
-
Attach the thick green/blue v.35 Serial cable to the port labeled "Serial 0" on the back of the access server.
-
Attach the DB25 to RJ45 converter hoods to the back of each modem .
-
Place the access server and the modems wherever you wish, remember though, you may at some point have 16 modems attached, along with the v.35 cable attachment from the serial 0 interface on the access server to the CSU/DSU. The access server can be rack mounted, or placed on a table or shelf. Try to plan ahead. Eventually you will have to make the physical switch of the CSU/DSU from the present equipment, to the 2511.
-
Once the physical placement of the new equipment has been planned, if practical, attach the male RJ45 ends of the Octal cables to the RJ45 ports on the back of the modems. You may wish to do this after you swap out the old equipment and put in the new.
-
At this point you are ready to switch from your present equipment to the Cisco. Power off and unplug the Cisco 2501 or 2502 presently in use. When you do this, access to MOREnet and the Internet will be lost.
-
Remove the v.35 Serial cable presently connected, from the back of the CSU/DSU.
-
Disconnect the LAN connection (either the Ethernet transceiver or Token Ring DB9 connector).
-
Remove this equipment to make room for the new access server and modems.
-
Attach your LAN connection to the back of the access server.
-
Attach the v.35 green/blue serial cable to the back of the CSU/DSU.
-
Plug in and power on the access server. In a few moments you should see your CSU/DSU return to a normal state.
-
Plug in and power on your modems.
Note: Before connecting your modems to the PSTN (Public Switched Telephone Network) complete the remainder of the configuration process. You don't want anyone calling while you are doing this, and if the number has been made available, someone will call!
Configure Cisco's 2500 Series Access Server IOS
After completing the hardware setup, configure the Cisco Internet-work Operating System (IOS). This procedure is more fully documented in the Access and Communication Server Configuration Guide, and the Access and Communication Server Command Reference, available on the UniverCD -ROM. In this guide we will explore only a few of the many features available.
Access the Cisco 2500 Series Operating System
Your access server was shipped pre-configured to operate just as your previous 2501 or 2502 model did before it was disconnected. You can access the router either by attaching an ASCII terminal to the console port of the router, or from a workstation on your local LAN via telnet. Telnet is the preferred method because it is likely you will use this method almost exclusively to communicate with the access server. Therefore we will not cover console port access here.
-
From any workstation on your network open your favorite telnet program and enter the IPaddress of the access server. This can be the ethernet or serial address of your router.
-
At the prompt type in the password shown in the line vty 0 4 section of the printed configuration file shipped with your access server. It will not be echoed back to your screen.
Note: For the remainder of this guide, bold constant width type indicates commands entered into the access server.
User Access Verification
Password:
MySchool>
The > sign at the end of the prompt indicates we are in EXEC mode.
The Cisco user interface is separated into a hierarchy of privilege levels. Each level provides access to a superset of commands that includes all commands from the previous level. In EXEC mode the configuration of the access server cannot be viewed or modified. We can, however, display who is using the access server:
MySchool>who
MySchool>who
Line User Host(s) Idle Location
1 tty 1 charley Async interface 0
* 18 vty 1 idle 0 128.206.206.126
MySchool>
From this prompt we can issue many other commands including a "?" which will list all commands available at this prompt. All commands except the "?" must be followed by ENTER. Your router has been shipped with 3 command levels:
EXEC (level 1) user mode or exec mode
ENABLE (level 15) enable mode
CONFIG Global configuration mode
Anyone with a valid user ID and password can use EXEC commands.
Only administrators should use the enable password to use ENABLE commands. This mode gives root access to the router.
CONFIG commands are used to edit your router’s configuration. This must be done from the enable mode:
MySchool>enable
Password:
MySchool#
Getting Around in Configuration Mode
Before attempting to modify any configuration parameters, it might be helpful to see the present set up. A distinction must be made between the running configuration and the configuration saved in nonvolatile RAM. When the access server is first powered up, it learns its expected behavior by reading the configuration file saved in non-volatile RAM. Once the access server has completed the boot process, you can modify the configuration.
Important: Any changes you make to the running configuration take affect immediately, but they are not automatically saved in nonvolatile RAM. You must execute a write memory command to save your changes.
The following command displays the current operating configuration:
MySchool#show runningconfig
This command displays the configuration stored in non-volatile RAM:
MySchool#show startupconfig
Just as there are multiple privilege command levels, Cisco's uner interface supports different types of configuration modes. For example, some commands are global in nature. That is, they affect the operation of the entire access server. Some commands may affect only a particular portion of the access server (for example, modem lines).
To enter global configuration mode:
MySchool#config terminal
Enter configuration commands, one per line. End with CNTL/Z
MySchool(config)#
Configuration commands entered from this prompt apply to the access server as a whole. To do any configuration modifications you must first enter global configuration mode:
To enter line configuration mode:
MySchool(config)#line 1 3
MySchool(configline)#
Configuration commands entered here affect lines 1 through 3. We could specify any one single line, or as we have done here, a range of lines.
To enter interface configuration mode:
MySchool(config)# interface Group-Async1
MySchool(configif)#
Commands entered in interface configuration mode apply to interfaces. Here we have chosen the first (of 16) available asynchronous interfaces.
Each port on the Cisco 2500 series Access Server (including the aux and console) is assigned a specific line number. 1 - 16 apply to the async modem ports. They may be referenced in a couple of ways:
The following command enters line configuration mode and any instructions here will apply to the first 3 modem lines:
MySchool#config terminal
Enter configuration commands, one per line. End with CNTL/Z.
MySchool(config)#line 1 3
MySchool(configline)#
The following command is equivalent:
Notice though the lines are referred to as tty lines. The issue here is to make a distinction between tty lines and vty lines. The term vty indicates we are applying commands to virtual terminal lines, which are created for incoming telnet sessions, not incoming calls.
MySchool#config terminal
Enter configuration commands, one per line. End with CNTL/Z.
MySchool(config)#line tty 1 3
MySchool(configline)#
The following command would apply to telnet connection lines:
It is unlikely you will encounter a need to configure vty lines. It is much more likely that you might want to clear a vty line if there is a hung session on it. In the following example, if you mistakenly put in tty 1 instead of vty 1, the user on line one will be disconnected:
MySchool#who
Line User Host(s) Idle Location
1 tty 0 services.more.net
* 2 tty 1 idle 3w1d 128.206.206.126
17 vty 0 idle 0 128.206.206.126
MySchool#clear line tty 2
[confirm]
[OK]
MySchool#who
Line User Host(s) Idle Location
1 tty 1 services.more.net
17 vty 0 idle 0 128.206.206.126
Other Commands You Need to Know
There is an additional section that shows examples of the most common commands you will need to use and the output from those commands where applicable. Any command can be shortened as long as it is unambiguous (for example, show configuration can be typed as sho conf, but not as sho con).
Specific Commands to Configure Your Modems
The following is a step-by-step example of configuring the access server and modems. The modems are configured using a reverse telnet that is explained later. Lines beginning with a "!" represent a comment, and can be used in the access server configuration file.
The following assumes that we will lock DTE speed, set hardware (RTS/CTS) flowcontrol, set carrier detect to reflect the actual carrier state, and set the modem to hang up on loss of DTR.
Note: 115200 is the maximum speed for the 25xx access servers, but 57600 is recommended.
Note: To "undo" or remove any configuration commands use the "no" form of the command. For example, to remove the flowcontrol hardware command use no flowcontrol hardware.
MySchool#config term
Enter configuration commands, one per line. End with CNTL/Z.
MySchool(config)#line 1 3
MySchool(configline)#speed 57600
MySchool(configline)#modem inout
MySchool(configline)#flowcontrol hardware
MySchool(configline)#end
MySchool#
Basic Modem Configuration Tasks
This section covers configuring your modems. It would be impossible to account for every situation, but this section should be enough to get your modem pool active.
Security on your Modems
To configure the access server use modem inout or modem riiscd (or modem dialin).
Turn Security off: Use modem inout to allow incoming and outgoing connections to the modem. You will need modem inout while configuring the modem.
Turn Security on: Use modem riiscd to allow incoming-only connections.
This is highly recommended unless you absolutely need to use the access server to dial out.
Cabling other than that recommended at the beginning of this section can cause modem control to fail since modem DCD may not be wired.
On the modem usually &C1 and &D2 control this function. This is often referred to as RS232 standard operation.
Configure Your Modems
Now configure each attached modem using a reverse telnet connection. From the access server prompt initiate another telnet session using the command:
MySchool#telnet x.x.x.x 20yy
where:
x.x.x.x is your primary ethernet address of your router, or gateway
20yy is the port number of your modem
MODEM 1 is PORT 2001
MODEM 2 is PORT 2002
Sample snap shot:
MySchool#telnet 204.184.26.254 2001 …. telnets you into a session with the modem.
AT Attention modem
OK … tells you that you are in a terminal session with the modem.
Enter your modem initialization string now…
AT ... see the back modem string pages for your proper command
To exit the modem you must use the Cisco escape sequence: CTL+SHT 6 immediately followed by x. Then disconnect you session by issuing a clear line 1 (or the number of your modem you were just configuring).
MySchool>enable
password: ernie
MySchool#telnet 204.184.26.254 2001
************************************************
* Welcome to our school’s modem pool *
* Enter your user ID and password as prompted. *
************************************************
AT
OK
AT&FS0=1&C1&D3&e1&E4&e15$BA0$SB115200&W0
OK
CTL SHT 6
x … If this is unsuccessful, disconnect your telnet session.
MySchool#clear line 1
confirm(y)
MySchool#
disconnecting active connection
MySchool#
NOTE: You can also use the disconnect command.
MySchool#disconnect
confirm(y)
MySchool#
disconnecting active connection
MySchool#
MySchool#telnet 204.184.26.254 2002
Do this for each of your modems
Specific Commands to Configure the Access Server Async Interfaces
Finally, in order to give users SLIP/PPP access, we need to configure what are referred to as Asynchronous interfaces. Each interface must provide an IP address for the remote user, and some environment settings for the remote workstation. Interfaces, regardless of their type, must be configured one at a time. We cannot specify a range of interfaces as we can with line configuration commands. There are many possibilities for configuration of async interfaces. This is the recommended configuration for allowing SLIP/PPP access to users. Each line is explained below.
MySchool#conf term
Enter configuration commands, one per line. End with CNTL/Z.
MySchool(config)# interface Group-Async1
MySchool(configif)#ip unnumbered Ethernet0
MySchool(configif)#ip tcp headercompression passive
MySchool(configif)#async mode interactive
MySchool(configif)#group-range 1 16
MySchool(configif)#peer default ip address pool ModemsIP
MySchool(configif)#ip local pool ModemsIP 204.185.54.127 204.185.54.142
MySchool(configif)#end
MySchool#write memory
ip unnumbered Ethernet0 - Whenever the unnumbered interface generates a packet (for example, for a routing update), it uses the address of the specified interface as the source address of the IP packet.
ip tcp headercompression passive - Header-compression will be determined by the calling workstation.
async mode interactive - Lets the user ask for an ip address using the slip command.
peer default ip address pool ModemsIP - This is the IP address pool that will be assigned to the calling workstation as a result of the slip default command. It should be an address from the pool of addresses assigned to your network.
Group-range 1 16 - This is the group of async lines you want this to apply to.
ip local pool ModemsIP x.x.x.x x.x.x.y - This defines the group of IP addresses you will use for the async lines. This assigns the group of addresses to the name ModemsIP.
Configure User Authentication on the Access Server
At this point you are ready to configure the Cisco 25xx Access Server authentication scheme. Some of you will be using the access server itself to provide user authentication. Others might choose to use xtacacs or tacacs+. This section will discuss each of these methods of user authentication.
Authentication using the 25xx Access Server
User authentication on the 25xx is easy in most cases. Users and their passwords are entered in global configuration mode and unless there are special circumstances, can be entered quickly and easily. In some cases there might be circumstances where some additional parameters can be associated with a specific user. There are of course some drawbacks to using this method. Users cannot change their own passwords, each one must be changed by the sysadmin. It is impractical to do this with large numbers of users. It would be very difficult to manage such an environment. There is also very little accounting function available under this method. The access server is simply not designed to store this kind of information.
To configure user authentication on the access server, telnet to your 25xx, login and enter admin mode. The following command is used to add users and passwords to the configuration:
MySchool#conf term
Enter configuration commands, one per line. End with CNTL/Z.
MySchool(config)#username bert password ernie
MySchool(config)#username charley password consulting
MySchool(config)#end
MySchool#sho run
Building configuration.
[text removed]
username bert password 7 104D080B11181D05
username charley password 7 011709035304131C24
[text removed]
Remove a User or Changing Their Password
If you need to remove a user or change their password you use the "no" version of the command. Since the user's password is encrypted you will not be able to view the password.
Remove the user and add them back in as a new user with a new password.
MySchool#conf term
Enter configuration commands, one per line. End with CNTL/Z.
MySchool(config)#no username bert
MySchool(config)#username bert password ernie2
MySchool(config)#end
MySchool#sho run
Building configuration.
The passwords are encrypted when displayed. The 7 indicates the encryption method and in this case is Cisco's proprietary encryption method. The only supported encryption methods are 0 for clear text (not recommended) and 7, the Cisco method.
In addition to simply assigning a username and password, this command can also associate an autocommand or an access-list with a specfic user. For example:
MySchool(config)#username bert password ernie
autocommand telnet x.x.x.x
In this case bert would automatically be telnetted to host x.x.x.x when he logged in to the access server.
TACACS/XTACACS
TACACS (Terminal Access Controller Access Control System) and originally was developed for the Department of Defense. XTACACS (extended TACACS), provides password checking, authentication and notification of user actions for security and accounting purposes, information about protocol translator and communication server use. This information is used in UNIX auditing trails and accounting files. Standard TACACS is rarely used any longer and we will focus only on XTACACS and TACACS+. What follows is a standard XTACACS configuration.
MySchool#conf term
Enter configuration commands, one per line. End with CNTL/Z.
MySchool(config)#tacacsserver host x.x.x.x
MySchool(config)#tacacsserver extended
MySchool(config)#tacacsserver notify connections
MySchool(config)#tacacsserver notify enable
MySchool(config)#tacacsserver notify logout
MySchool(config)#tacacsserver notify slip
MySchool(config)#end
MySchool#
tacacsserver host x.x.x.x - specify the ip address of the machine running the tacacs server. Multiple TACACS servers can be specified simply by duplicating the command for each server host machine.
tacacsserver extended - use the extended TACACS command set
tacacsserver notify connections - send notification to the TACACS server of a connection attempt to another host
tacacsserver notify enable - send notification to the TACACS server that a user has entered enable mode
tacacsserver notify logout - send notification to the TACACS server that a user has logged out
tacacsserver notify slip - send notification to the TACACS server that a user has requested a SLIP connection.
Each of these commands is fully documented in the Access and Communications Server Command Reference available on the UniverCD cd-rom.
As you are aware by now, in order for TACACS to function, there must be a TACACS daemon running on a host somewhere on your network. At the time of this writing there are four platforms for running XTACACS; Unix (most any flavor), Windows (all flavors), and a pre-release for Netware 4.1 and 3.12. I have used the Netware and Windows versions in addition to the Unix version.
The Windows version offers little advantage at this time except accounting and a GUI interface. Using the access server itself will be just as effective. The Windows version is shareware and is reasonably priced.
The WindowsNT version will authenticate users using the NT user tables.
The Netware version seems to provide the most benefit. It is, however, limited in its ability to distiguish one group from another, although you can specify which users will be granted access and which will not. In my very limited testing I saw no way to specify one group authorized for slip and another group not authorized, for example. This is also commercial software and lists for $250 per license with a discount for multiple licenses.
TACACS+
To my knowledge, there is only one stable platform for TACACS+, and that is the UNIX platform (most any flavor). There are TACACS+ daemons in beta testing for other platforms. AAA/TACACS+ provides more detailed accounting information as well as more administrative control of authentication and authorization processes. The following is a sample of the commands used to configure the access server for aaa/tacacs+:
!Tell the commserver to use the aaa/tacacas+ protocol
aaa new-model
!Set up the default login process. He will use this process if no other login !rocess is specified
aaa authentication login default tacacs+ enable
!Set up another login process
aaa authentication login telnet tacacs+ line
!Do not allow any process to take place unless accounting records can be !ritten to the tacacs+ server
aaa accounting exec start-stop tacacs+
aaa accounting commands 0 start-stop tacacs+
aaa accounting network start-stop tacacs+
aaa accounting system start-stop tacacs+
!Tell him about the tacacs+ server
tacacs-server host 198.209.250.153
tacacs-server key q1Fc1w3YWDd.c
The aaa authentication login commands specify the process the commserver will use to authenticate users. If no login process is specified on a line the the default will be used, which is to first ask the tacacs+ server, and if a deny is returned allow the enable password to be used to gain access. If however a login process is specified for a line like this:
line vty 0 4
password poppins
login authentication telnet
logout-warning 240
absolute-timeout 5
The telnet process is used so if there is no response from a tacacs+ server, the user can gain access using the assigned line password, "poppins".
AAA/TACACS+ commands are fully documnted in the Access and Communication Server Configuration Guide, and the Access and Communication Server Command Reference, available on the UniverCD cd-rom. As of this writing MOREnet is using XTACACS in our production environment. There are plans however to move to AAA/TACACS+ in the near future.
Configure a UNIX TACACS/XTACACS or TACACS+ Server
It would once again be impossible to document every possible daemon configuration issues in this discussion. Here I will try to provide some tips for setting up a TACACS daemon in a Unix environment. In the interest of brevity I must assume the reader has a working knowledge of the Unix environment and shell commands.
It is not particularly difficult to set up an XTACACS or TACACS+ daemon. FTP the source code from one of the resources listed below and copy the file wherever you want to put it (probably /usr/local/src). Read the README and MAKEFILE notes. You will need to provide the full path to where ever you want the logs to be (the default is fine). Check and see if any CFLAGS need to be set in your environment. Go ahead and "make" the binary.
For the XTACACS daemon you will want to run the daemon as a stand alone process. I suggest not running it from inetd. You will almost certainly want to run it like the following:
/usr/local/bin/xtacacsd -s -l
The -s switch means "I am a standalone daemon" and the -l switch enables logging. In addition the -h switch instructs the daemon to create a separate wtmp file for each host that makes a request of the daemon.
Finally you might also want to use some of the configuration file options that are available to you in which case you will want to start the daemon with somwthing like:
/usr/local/etc/xtacacsd -s -l -c /usr/local/etc/xtacacsd-conf
The configuration file then has many options that can be controlled from it. The most common use for MOREnet is to allow some users to use SLIP access and deny others. The following line accomplishes this:
## 1/1/96 ESK - Allow SLIP groups slip access through terminal server
GROUP 10 | HOST all | slipon permit |
GROUP 1011 | HOST all | slipon permit |
GROUP 1501 | HOST all | slipon permit |
GROUP 1506 | HOST all | slipon permit |
GROUP 1511 | HOST all | slipon permit |
GROUP 1516 | HOST all | slipon permit |
GROUP 1521 | HOST all | slipon permit |
GROUP 1526 | HOST all | slipon permit |
GROUP all | HOST all | slipon deny |
Notice the last line. It means deny everyone not included in the above groups SLIP access. The reason this is necessary is because the daemon is simply going down the list looking for a match. If we left this line off, any one could access SLIP. Read the XTACACS documentation for further details.
TACACS+ is an entirely different beast, although at least part of its job is the same. TACACS+ uses its own configuration file to store user and group attributes. This allows for much "finer" control of the access process but has one serious drawback. When a user changes his password, the system updates /etc/passwd (probably/maybe). Since TACACS+ uses its own password file the administrator must come up with a way to update this file. Utilities are included to port passwd(5) type files to TAC+ files, bnut it means that this process must be done periodically. Unitil the TAC+ password file is updated, the user will be unable to login using his new password. S/he will still be able to use the old password.
One big advantage to TACACS+ is the ability to return an "autocommand" based on user or group etc. As mentioned earlier when using the 25xx access server itself for configuration, we could associate a "username" with a command. The same thing applies here, but in this case we have the power to reasonably manage large numbers of users at this level.
Another advantage of TACACS+ is its level of accounting. The TAC+ daemon writes ascii text files instead of wtmp files that can be difficult to manage. The user is free to write his own scripts (eg. Perl scripts) to use the data however he sees fits. The daemon however will write an "accounting" file of its own, in addition to the usual logfile. This file is also in ascii text and is easy to interpret.
Troubleshoot Modem Pool Problems
There are any number of things that can (and almost surely will) go wrong during modem pool operation. This section will provide tips for troubleshooting and correcting modem pool problems.
The following commands are useful for watching modem activity. Each command is followed by a sample of the command's output. Each of these commands is fully documented in the UniverCD shipped with your access server.
Show Line Status Command
MySchool#sho line 1
Tty Typ Tx/Rx A Modem Roty AccO AccI Uses Noise Overruns
I 1 TTY 57600/57600 - inout - - - 14 15 0/0
Line 1, Location: " ", Type: " "
Length: 24 lines, Width: 80 columns
Baud rate (TX/RX) is 57600/57600, no parity, 1 stopbits, 8 databits
Status: No Exit Banner
Capabilities: Hardware Flowcontrol In, Hardware Flowcontrol Out
Modem Callout, Modem RI is CD, Line usable as async interface
Output non-idle
Modem state: Idle
Special Chars: Escape Hold Stop Start Disconnect Activation
^^x none - - none
Timeouts: Idle EXEC Idle Session Modem Answer Session Dispatch
0:10:00 2:00:00 4:00:00 not set
Session idle time reset by output.
Session limit is not set.
Time since activation: never
Editing is enabled.
History is enabled, history size is 10.
Full user help is disabled
Allowed transports are telnet rlogin. Preferred is telnet.
No output characters are padded
No special data dispatching characters
Modem hardware state: noCTS noDSR DTR RTS
Group codes: 0
TCP/IP header compression enabled
Using the Debug Command to Debug Your Modems
MySchool#term mon
MySchool#debug modem
TTY12: DSR was dropped
TTY12: Quickly dropping DTR
tty12: Modem: READY->CARDROP
TTY12: Line reset
TTY12: Modem: CARDROP->HANGUP
TTY12: dropping DTR, hanging up
tty12: Modem: HANGUP->IDLE
TTY12: restoring DTR
TTY15: Line reset
TTY15: Modem: READY->HANGUP
TTY15: dropping DTR, hanging up
tty15: Modem: HANGUP->IDLE
MySchool#undebug all
Debuggin Your TACACS Authentication
MySchool#term mon
MySchool#debug tacacs
TAC: Sent notification type LOGOUT (7) to 204.184.50.200, Id 387
TAC: received extended ANSWER (2) from 204.184.50.200, via Ethernet0 Id 387
TAC: ack received for type (LOGOUT) 7 to 204.184.50.241, Id 387
TAC: Sent notification type LOGOUT (7) to 204.184.50.200, Id 2538
TAC: received extended ANSWER (2) from 204.184.50.200, via Ethernet0 Id 2538
TAC: ack received for type (LOGOUT) 7 to 204.184.50.241, Id 2538
MySchool#undebug all
Note: Be sure to issue the undebug all command to stop debugging.
Saturday, November 8, 2008
Article : 2002 - All Hospital Network Down - Reason for disaster :)
All systems down
By Scott Berinato
Among the 30-odd CIOs who serve Boston's world-famous health-care institutions, John Halamka is a star among stars. He has been CIO of the CareGroup health organization and its premier teaching hospital—the prestigious Beth Israel Deaconess Medical Center—since 1998. He helps set the agenda for the Massachusetts Health Data Consortium, a confederation of executives that determines health-care data policies for New England.
Until 2001, the 40-year-old Halamka also worked as an emergency room physician, but he gave that up to take on the additional responsibilities of being CIO of Harvard Medical School in 2002. However, as a globally recognized expert on mushroom and wild plant poisonings, he is still called when someone ingests toxic flora.
All of this has earned Halamka a considerable measure of renown. For two years running, InformationWeek named Halamka's IT organization number one among hospitals in its yearly ranking of innovative IT groups. In September 2002, CareGroup was ranked 16th on InformationWeek's list of 500.
Two months later, Beth Israel Deaconess experienced one of the worst health-care IT disasters ever. Over four days, Halamka's network crashed repeatedly, forcing the hospital to revert to the paper patient-records system that it had abandoned years ago. Lab reports that doctors normally had in hand within 45 minutes took as long as five hours to process. The emergency department diverted traffic for hours during the course of two days. Ultimately, the hospital's network would have to be completely overhauled.
This crisis struck just as health-care CIOs are extending their responsibilities to clinical care. Until recently, only ancillary systems like payroll and insurance had been in the purview of the CIO. But now, in part because of Halamka and his peers, networked systems such as computerized prescription order entry, electronic medical records, lab reports and even Web conferencing for surgery have entered the life of the modern hospital. These new applications were something for health-care CIOs to boast about, and Halamka often did, even as the network that supported the applications was being taken for granted.
"Everything's the Web," Halamka says now. "If you don't have the Web, you're down."
Until last Nov. 13, no one, not even Halamka, knew what it really meant to be down. Now, in the wake of the storm, the CIO is calling it his moral obligation to share what he's learned.
"I made a mistake," he says. "And the way I can fix that is to tell everybody what happened so they can avoid this."
Sitting in his office three weeks after the crash, Halamka appears relaxed and self-possessed. There's another reason he's opening up, talking now about the worst few days of his professional life at CareGroup. "It's therapeutic for me," he says, and then he begins reliving the disaster.
Wednesday: The network flaps
On Nov. 13, 2002, a foggy, rainy Wednesday, Halamka was alone in his office at Beth Israel when he noticed the network acting sluggishly. It was taking five or 10 seconds to send and receive e-mail. Around 1:45 p.m., he strolled over to the network team to find out what was up.
A few of his 250 IT staff members, who range from low-level administrators to senior application developers, had already noted the problem. They told him not to worry. There was a CPU spike—a sudden surge in traffic. RCA, one of the core network switches, was getting pummeled. From where, they didn't know. It might have to do with a consultant who was working on RCA, preparing it for a network remediation project.
"We happened to have had a guy in there," recalls Russell Rusch of Callisma, the company leading the remediation project. "We knew the hospital had had similar incidents in the past few months." Those previous CPU spikes lasted anywhere from 15 minutes to two hours, he says. Then they worked themselves out. Like indigestion.
Halamka's team decided to begin shutting down virtual LANs, or VLANs. They would turn off switches to isolate the source of the problem, much in the same way one would go around a house shutting off lights to find out which one was buzzing. Halamka thought the plan sounded reasonable.
It was a mistake.
Shutting switches forced other switches to recalculate their traffic patterns. These calculations were so complex that those switches gave up doing everything else.
Traffic stopped. The network was down.
Within 15 minutes, by 2 p.m., the team reversed course and turned all the switches back on. A sluggish network, they figured, was preferable to a dead one.
For the rest of the day and into the night, the network flapped—a term Halamka uses to describe the network's state of lethargy dotted by moments of availability and, more often, spurts of dead nothing. The team searched for the cause. Around 6 p.m., when most of the doctors, nurses, staff and students left, the network settled down. Finally, at 9 p.m., the IT staff found its gremlin: a spanning tree protocol loop.
Spanning tree protocol is like a traffic cop. Data arrives at a switch and asks spanning tree for directions. Say, from John's server to Mary's desktop. Spanning tree calculates the shortest route. It then blocks off every other possible route so that the data will go straight to its destination without having to make decisions at other crossroads along the way.
But spanning tree will look only as far out as seven intersections. Should data reach an eighth intersection, called a hop in networking, it will lose its way. Often, it will drive itself into a loop. This clogs the network in two ways. First, the looped traffic itself gums up the works. Then, other switches start to use their computing horsepower to recalculate their spanning trees—to make up for the switch that is directing traffic in a loop—instead of directing their own traffic.
That's what happened at Beth Israel Deaconess. On Wednesday, a researcher uploaded data into a medical file-sharing application, and it looped. The data was several gigabytes, so it clogged the pipes. Then, when Halamka's team turned off a switch at 1:45 p.m., it was as if one cop closed an intersection and every other cop stopped traffic in all directions to figure out alternate routes.
Halamka's team now knew what happened, if not where it happened. Standard troubleshooting protocol for spanning tree loops calls for cutting off redundant links on the network. "What you're doing is eliminating potential spots where there are too many hops, and creating one path from every source to every destination," Callisma's Rusch says. "It might make for a slower environment"—without backup—"but it should make for a stable environment."
"We cut the links," Halamka says. "It seemed to work. We went home feeling great. We had figured it out."
Thursday: Clogged arteries
Hospitals come alive early. By 7 a.m., doctors and nurses started to send some of Beth Israel Deaconess's 100,000 daily e-mails. The pharmacy began filling prescriptions, transferring the first bits of the 40 terabytes that traverse the network daily. Some of the 3,000 daily lab reports were beginning to move.
By 8 a.m., the network again started acting as if it were flying into a headwind. Halamka realized the network had settled down the night before only because hardly anyone was using it. When the workday began in earnest, CPU usage spiked. The network started flapping. The problem hadn't been fixed.
Halamka's team scrambled to find other possible sources of the trouble. One suspect was CareGroup's network of outlying hospitals in Cambridge, Needham, Ayer and elsewhere in Massachusetts. They operated as a distinct network that plugged into Beth Israel Deaconess. The community hospitals' network was sluggish, and a billing application wasn't working, according to Jeanette Clough, CEO of Mount Auburn Hospital in Cambridge, which serves as the hub for the outlying hospitals' network.
The easiest thing to do would be to cut the links, eliminating the potential for spanning tree loops. But that would isolate the outlying hospitals. Instead, the IS team, along with Callisma engineers, chose a more complex option. They would try converting from switching to routing between the core network and the outlying hospitals. That would eliminate spanning tree issues while keeping those hospitals connected.
They tried for seven hours, and, for arcane reasons that have to do with VLAN Trunking Protocol (VTP), they never got the routing to work. The network flapped all day.
Around midmorning, as Halamka was explaining the routing strategy to CareGroup executives in an ad hoc meeting, a patient, an alcoholic in her 50s, was admitted to Beth Israel Deaconess's ICU. Dr. Daniel Sands, a primary care physician and director of the hospital's clinical computing staff, saw her. She had what Sands calls "astounding electrolyte deficiencies," a problem common to people who drink their meals. In fact, Sands says, "It was incredible she was alive.
"I needed to be careful with this woman. I needed to try treatments based on lab reports and then monitor progress and adjust as I went," recalls Sands. "But all of a sudden, we couldn't operate like that. Usually I get labs back in less than an hour; they were taking five hours, and here I have a patient who could die. I was scared." (The patient would survive.)
At 4 p.m., Halamka met with a minicrisis team that included the head of nursing, the heads of the lab and the pharmacy, and hospital COO Dr. Michael Epstein. "Even then," Halamka says, "I'm still saying, 'We're one configuration change away,' and my assumption is things will be back up soon."
But his team was tense and frustrated. CareGroup's help desk had been flooded with calls. They were hearing everything from "I can't check my e-mail" to "I don't know if the blood work I just requested went through."
At 3:50 p.m., Beth Israel closed its emergency room. It stayed closed for four hours, until 7:50 p.m., according to Massachusetts Department of Public Health documents.
It was at the 4 p.m. meeting that COO Epstein says he realized "this was more than a garden-variety down-and-up network." Clinical users, like Sands, were signaling that they were worried. Epstein and Halamka, along with hospital executives and network consultants, decided to take extreme measures. They called Cisco Systems, the hospital's San Jose, Calif.-based equipment and support vendor. Cisco responded by triggering its Customer Assurance Program (CAP), a bland name that belies how rare and how serious CAPs are. CAP means Cisco commits any amount of money and every resource available until a crisis is resolved.
CAP was declared shortly after 4 p.m. By 6 p.m., a local CAP team from nearby Chelmsford, Mass., had set up a command center at the hospital and initiated "follow the sun" support—meaning additional staff at Cisco's technical assistance centers would be plugged in to the crisis until their workday ended, when they'd hand off support to a similar group a few time zones behind them.
First, the CAP team wanted an instant network audit to locate CareGroup's spanning tree loop. The team needed to examine 25,000 ports on the network. Normally, this is done by querying the ports. But the network was so listless, queries wouldn't go through.
As a workaround, they decided to dial in to the core switches by modem. All hands went searching for modems, and they found some old US Robotics 28.8Kbps models buried in a closet. Like musty yearbooks pulled from an attic, they blew the dust off them. They ran them to the core switches around Boston's Longwood medical area and plugged them in. CAP was in business.
An outmoded network
By 9 p.m., they had pinpointed the problematic spanning tree loop. The Picture Archive Communication System (PACS) network, for sharing high-bandwidth visual files and other clinical data, was 10 hops away from the closest core network switch, three too many for spanning tree to handle.
And that's when the dimensions of the problem fully dawned on the team members: They were struggling with an outmoded network. In September 2002, Halamka had hired Callisma's Rusch to audit CareGroup's infrastructure. When Rusch finished, he told Halamka, "You have a state-of-the-art network—for 1996."
Halamka's network was all Layer 2 switches with no Layer 3 routing. Switching is fast, inexpensive and relatively dumb, and it relies on spanning tree protocol. Routing is more expensive but smarter. Routers have quality-of-service throttles to control bandwidth and to isolate heavy traffic before it overwhelms the network. State-of-the-art networks in 2002 have routing at their core.
In 1996, CareGroup's network was Beth Israel Hospital, and at its core was a switch called Libby030. In October of that year, the hospital merged with Deaconess Hospital. Deaconess's network was plugged into Libby030.
Other systems were tacked on in the same way. In 1998, CareGroup connected PACS to what used to be Deaconess Hospital. A year later, CareGroup linked a new data center and its two core switches (RCA and RCB) to Libby030. There would be a fourth core switch added and a skein of redundant links, but Libby030 remained the main outlet. Halamka now understands that this was a "network of extension cords to extension cords. It was very fragile," he says.
To fix the problem, the CAP team decided to put a Cisco 6509 router between the core network and PACS, eliminating spanning tree protocol and its seven-hop limitation. (The 6509 also has switching capabilities, so the team decided to kill three switches inside PACS and use the 6509 for that too.)
Soon after 9 p.m., a Boeing 747 with a Cisco 6509 on board left Mineta International Airport in San Jose bound for Boston's Logan International Airport.
The local CAP team spent the night rebuilding the PACS network, a feat Halamka talks about with a fair bit of awe: The first time around, PACS took six months to build.
After working through the night, the team was momentarily disheartened Friday morning to see that, despite PACS being routed, the network was still saturated. But they rebooted Libby030 and another core switch, which brought out the smiles.
"We rebooted and things looked pretty," Halamka says.
Friday: Back to paper
By 8 a.m., the network started to flap again.
At 10 a.m., Halamka and COO Epstein decided to shut down the network and run the hospital on paper. The decision turned out to be liberating.
"We needed to stop bothering the devil out of the IT team," says Epstein.
Shutting down the network also freed Sands and the hospital's clinicians. Some had already given up on the computers but felt guilty about it. But "once the declaration came that we were shutting down the network, we felt absolved of our guilt," Sands recalls.
The first job in adapting to paper is to find it: prescription forms, lab request forms. They had been tucked away and forgotten. And many of the newer interns had never used them before. On Friday, they were taught how to write prescriptions. When Sands had to write one, it was his first in 10 years at CareGroup. "When I do this on computer, it checks for allergy complications and makes sure I prescribe the correct dosage and refill period. It prints out educational materials for the patient. I remember being scared. Forcing myself to write slowly and legibly."
At noon, Epstein came in to lend a hand ... and walked into 1978. Epstein worked the copier, then sorted a three-inch stack of microbiology reports and handed them to runners who took them to patients' rooms where they were left for doctors. (There were about 450 patients at the hospital.)
In time, the chaos gave way to a loosely defined routine, which was slower than normal and far more harried. The pre-IT generation, Sands says, adapted quickly. For the IT generation, himself included, it was an unnerving transition. He was reminded of a short story by the Victorian author E.M. Forster, "The Machine Stops," about a world that depends upon an ?ber-computer to sustain human life. Eventually, those who designed the computer die and no one is left who knows how it works.
"We depend upon the network, but we also take it for granted," Sands says. "It's a credit to Halamka that we operate with a mind-set that the computers never go down. And that we put more and more critical demands on the systems. Then there's a disaster. And you turn around and say, Oh my God."
Halamka had become an ad hoc communications officer for anyone looking for information. Halamka was the hub of a wheel with spokes coming in to him from everywhere—the CAP team, executive staff, clinicians and the outlying hospitals. Halamka leaned on his emergency room training at the Harbor-UCLA Medical Center in Los Angeles during the height of gang violence in the '90s. Rule one: Stay calm and friendly.
"But I'll be honest, 48 hours into this, with no sleep, the network's still flapping, I had a brave face on, but I was feeling the effects," Halamka recalls. "I was feeling the limitations of being a human being. You need sleep, downtime. You need to think about shifts, or humans will despair."
He found himself dealing with logistics that had never occurred to him: Where do we get beds for a 100-person crisis team? How do we feed everyone? He improvised.
"You don't know the details you're missing in your disaster recovery plan until you're dealing with a disaster," he says. For example, the paper plan was, in essence, the Y2K plan. Besides the fact that it was dated, it didn't address this kind of disaster.
Recovery plans are usually centered on lost data or having backups for lost data, or the integrity of data. At Beth Israel Deaconess, the data was intact. It was just inaccessible.
That led to Halamka's chief revelation: You can't treat your network like a utility.
"I was focusing on the data center. And storage growth. After 9/11, it was backup and continuance. We took the plumbing for granted. We manage the life cycle of PCs, but who thinks about the life cycle of a switch?"
This is a valuable insight. Networks indeed have gotten less attention than applications. But at the same time, Callisma's Rusch says, he hadn't seen a network as archaic as Beth Israel's in several years. "Many have already gotten away from that 1996 all-switched model," he says. "There are probably a couple of others like this out there."
Others agree with Rusch's assessment. "I think the danger is people start thinking the whole health-care IT industry is flawed and a train wreck waiting to happen," says the CIO of another hospital. "It's not. We all watched the heroic effort they made over there, but we're not standing around the water cooler talking about how nervous we are this will happen to us. We've had these issues. They scared us enough a few years ago that we took care of the architecture problem."
Halamka retreated to his office late Friday night. He lay down on the floor, pager in one hand, cell phone in the other, and fell asleep for the first time in two days. Two hours later, his cell phone rang.
Saturday: Helplessly hoping
Half awake, Halamka heard a staffer tell him they had found two more spanning tree errors, one at a facility called Research North and one in cardiology. Both had eight hops, one too many. They planned to cut the redundant links and move the traffic to the core network.
No one knew for sure how severely this would tax poor Libby030 and its counterparts. The team decided to build a redundant core with routing infrastructure as a contingency plan that would bring CareGroup out of 1996 and into 2002 in terms of its network.
At 8 a.m., two more Cisco 6509 routers (with switching capabilities) arrived from San Jose. Three hours before that, a trio of Cisco engineers from Raleigh, N.C., landed in Boston. They spent all day building a redundant network core.
Sands felt uncomfortable doing rounds that morning. "Patients sort of expect you to know their histories," he says. "But without that dashboard of information I'd get from the computer, I had to walk up to patients I had treated before and ask basic questions like, What allergies do you have? Even if I thought I remembered, I didn't trust my memory. It was embarrassing, and I was worried."
Progress on the network was slow. No one wanted to suggest that the current tack—building the redundant network core while severing the redundant links—was definitely the answer. At 9 a.m., Halamka met with senior management, including CareGroup CEO Paul Levy. "I can't tell you when we'll be functioning again," Halamka confessed.
Admitting helplessness is not part of Halamka's profile. "You never catch John saying, I'm scared, or I messed up," says one of his peers from the Health Data Consortium. "This had to be hard for him."
"When John told us he couldn't tell us when we'd be up, we stopped having him as part of our twice-a-day reports," Epstein recalls. The intent was to free Halamka from his communications duties so that he could focus on the problem. But Epstein was also becoming frustrated. He recalls thinking that "we didn't want to keep sending out memos to the staff that said, Just kidding, we're still down."
"If I had felt, in the heat of the battle, that someone could have done a better job than me, if I felt like I was a lesion, then I would have stepped aside," Halamka says. "At no time did I think this, and at no time was I fearful for my job. Am I personally accountable for this whole episode? Sure. Absolutely. Does that cause emotional despair? Sure. But I had to fix it."
Saturday night, with the redundant core in place, Halamka turned on the network. It hummed. There was clapping and cheering and backslapping among the team, which had grown to 100. Halamka passed around bottles of Domain Chandon champagne that his wife had bought at Costco. Then he went home.
At 1 a.m., his pager woke him.
Another CPU spike.
Sunday: And on the fifth day, Halamka rested
The problem was simple: A bad network card in RCB, one of the core switches. They replaced the card. Halamka went back to sleep.
Beep. 6 a.m. This time, it was a memory leak in one of the core switches. The CAP team quickly determined the cause: buggy firmware, an arcane VLAN configuration issue. They fixed it.
All day, the team documented changes. Halamka refused to say the network was back, even though it was performing well. "Let us not trust anyone's opinion on this," he recalls thinking. "Let us trust the network to tell us it's fine by going 24 hours without a CPU spike."
Monday: Back in business
Halamka arrived at his office at 4 a.m., nervous. He launched an application that let him watch the CPU load on the network. It reads like a seismograph. Steep, spiky lines are bad, and the closer together they are, the nastier the congestion. At one point on Thursday, the network had been so burdened that the lines had congealed into thick bars.
Around 7:30 a.m., as the hospital swung into gear, Halamka stared at the graph, half expecting to see the steep, spiky lines.
They never came. At noon, Halamka declared "business as usual." The crisis was over. It ended without fanfare, Halamka alone in his office. The same way it had started.
Friday, November 7, 2008
Thursday, November 6, 2008
Dil Haarey - Jal
Ankhoon Say Jaab Bhi Ho Teri Ankhein Judaa
G C D
Maan Say Jaab Bhi Ho Tera Maan Khafaa
Em D
Tou Jee Pukarey ..... Pukarey Tujhey
Em D
Tou Maan Ghabraaye ..... Bulayee Tujhey
G
Kay Dil Harey Pukarey Tujhey
C D
Maan Ja Ray Mana Laye Mujehy
G
Sun Pyareey Lagaley Galeyy
C D
Kho Ja Ray Tou Gaaley Sung Ray
G C D
Tere Bina Zindagi Lagey Ek Sazaa
G C D
Tera Saath Ho Yeh Sazaa Lagey Ek Jazaa
Em D
Tou Barhtey Jayein Yeh Rasety Sareyy
Em D
Duniyaa Pichey Hum Hein Aagey
G
Kay Dil Harey Pukarey Tujhey
C D
Maan Ja Ray Mana Laye Mujehy
G
Sun Pyareey Lagaley Galeyy
C D
Kho Ja Ray Tou Gaaley Sung Ray
Sunday, November 2, 2008
List of Protocols - on different layers of OSI model
* ADSL Asymmetric digital subscriber line
* ISDN Integrated Services Digital Network
* PDH Plesiochronous Digital Hierarchy
o T-carrier (T1, T3, etc.)
o E-carrier (E1, E3, etc.)
* RS-232, a serial line interface originally developed to connect modems and computer terminals
* SDH Synchronous Digital Hierarchy
* SONET Synchronous Optical NETworking
* Modem standards/ITU V-Series Protocols used to communicate between analog modems over voice telephone lines.
* Various Ethernet physical layers
Layer 2 protocols (Data Link Layer)
* ARCnet
* CDP Cisco Discovery Protocol
* DCAP Data Link Switching Client Access Protocol
* Dynamic Trunking Protocol
* Econet
* Ethernet
* FDDI Fiber Distributed Data Interface
* Frame Relay
* HDLC High Level Data Link Control
* IEEE 802.11
* IEEE 802.16
* LocalTalk
* L2F Layer 2 Forwarding Protocol
* L2TP Layer 2 Tunneling Protocol
* LAPD Link Access Procedures on the D channel
* LLDP Link Layer Discovery Protocol
* LLDP-MED Link Layer Discovery Protocol - Media Endpoint Discovery
* PPP Point-to-Point Protocol
* PPTP Point-to-Point Tunneling Protocol
* Q.710 Simplified Message Transfer Part
* NDP Neighbor Discovery Protocol
* SLIP Serial Line Internet Protocol (obsolete)
* StarLan
* STP Spanning Tree Protocol
* Token ring
* VTP VLAN Trunking Protocol
Layer 2+3 protocols
* ATM Asynchronous Transfer Mode
* Frame relay, a simplified version of X.25
* MPLS Multi-protocol label switching
* X.25
* ARP Address Resolution Protocol
* RARP Reverse Address Resolution Protocol
Layer 1+2+3 protocols
* MTP Message Transfer Part
* NSP Network Service Part
Layer 3 protocols (Network Layer)
* CLNP Connectionless Networking Protocol
* EGP Exterior Gateway Protocol
* EIGRP Enhanced Interior Gateway Routing Protocol
* ICMP Internet Control Message Protocol
* IGMP Internet Group Management Protocol
* IGRP Interior Gateway Routing Protocol
* IPv4 Internet Protocol version 4
* IPv6 Internet Protocol version 6
* IPSec Internet Protocol Security
* IPX Internetwork Packet Exchange
* MPLS Multiprotocol Label Switching
* SCCP Signalling Connection Control Part
Layer 3 protocols (Network Layer management)
* IS-IS Intermediate system to intermediate system
* OSPF Open Shortest Path First
* BGP Border Gateway Protocol
* RIP Routing Information Protocol
Layer 3.5 protocols
* HIP Host Identity Protocol
Layer 3+4 protocols
* Xerox Network Services
Layer 4 protocols (Transport Layer)
* AHAH Authentication Header over IP or IPSec
* ESPESP Encapsulated Security Payload over IP or IPSec
* GRE Generic Routing Encapsulation for tunneling
* IL Originally developed as transport layer for 9P
* SCTP Stream Control Transmission Protocol
* Sinec H1 for telecontrol
* SPX Sequenced Packet Exchange
* TCP Transmission Control Protocol
* UDP User Datagram Protocol
Layer 5 protocols (Session Layer)
* 9P Distributed file system protocol developed originally as part of Plan 9
* NCP NetWare Core Protocol
* NFS Network File System
* SMB Server Message
Layer 7 protocols (Application Layer)
* AFP, Apple Filing Protocol
* BACnet, Building Automation and Control Network protocol
* BitTorrent, A peer-to-peer file sharing protocol
* BOOTP, Bootstrap Protocol
* Diameter, an authentication, authorization and accounting protocol
* DICT, Dictionary protocol
* DNS, Domain Name System
* DHCP, Dynamic Host Configuration Protocol
* ED2K, A peer-to-peer file sharing protocol
* FTP, File Transfer Protocol
* Finger, which gives user profile information
* Gnutella, a peer-to-peer file-swapping protocol
* Gopher, a hierarchical hyperlinkable protocol
* HTTP, HyperText Transfer Protocol
* IMAP, Internet Message Access Protocol
* IRC, Internet Relay Chat protocol
* ISUP, ISDN User Part
* Jabber, an instant-messaging protocol
* LDAP Lightweight Directory Access Protocol
* MIME, Multipurpose Internet Mail Extensions
* MSNP, Microsoft Notification Protocol (used by Windows Live Messenger)
* MAP, Mobile Application Part
* NetBIOS, File Sharing and Name Resolution protocol - the basis of file sharing with Windows.
* NNTP, News Network Transfer Protocol
* NTP, Network Time Protocol
* NTCIP, National Transportation Communications for Intelligent Transportation System Protocol
* POP3 Post Office Protocol Version 3
* RADIUS, an authentication, authorization and accounting protocol
* Rlogin, a UNIX remote login protocol
* rsync, a file transfer protocol for backups, copying and mirroring
* RTP, Real-time Transport Protocol
* RTSP, Real-time Transport Streaming Protocol
* SSH, Secure Shell
* SISNAPI, Siebel Internet Session Network API
* SIP, Session Initiation Protocol, a signaling protocol
* SMTP, Simple Mail Transfer Protocol
* SNMP, Simple Network Management Protocol
* SOAP, Simple Object Access Protocol
* TUP, Telephone User Part
* Telnet, a remote terminal access protocol
* TCAP, Transaction Capabilities Application Part
* TFTP, Trivial File Transfer Protocol, a simple file transfer protocol
* WebDAV, Web Dist Authoring and Versioning