Asyncio code not returning multiple results for nextCmd

[ajwillo]

Good Evening,

I am migrating a script to be asynchronous and am using the asyncio library with pysnmp and what looks to be almost identical code between the synchronous and non synchronous script it pulling a lot of data for the sync code but what seems like only 1 entry for the async code.

using pysnmp==4.4.8

async walk:

   @asyncio.coroutine
   def snmp_walk(ip, oid, snmp_user, snmp_auth, snmp_priv):
       results=[]
       snmpEngine = SnmpEngine()
       errorIndication, errorStatus, errorIndex, varBinds = yield from nextCmd(
                               snmpEngine,
                               UsmUserData(snmp_user, snmp_auth, snmp_priv,
                               authProtocol=usmHMACSHAAuthProtocol,
                               privProtocol=usmAesCfb128Protocol),
                               UdpTransportTarget((ip, 161)),
                               ContextData(),
                               ObjectType(ObjectIdentity(oid)),
                               lexicographicMode=False
                           )
       if errorIndication:
           print(errorIndication)
       elif errorStatus:
           print('%s at %s' % (
               errorStatus.prettyPrint(),
               errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
           )
                 )
       else:
           for varBind in varBinds:
               results.append(' = '.join([x.prettyPrint() for x in varBind]))
       snmpEngine.transportDispatcher.closeDispatcher()
       print(results)
       return results

sync walk:

from pysnmp.hlapi import *
    def snmp_walk(ip, oid, snmp_user, snmp_auth, snmp_priv):
        results=[]
        for (errorIndication,
            errorStatus,
            errorIndex,
            varBinds) in nextCmd(SnmpEngine(),
                                UsmUserData(snmp_user, snmp_auth, snmp_priv,
                                authProtocol=usmHMACSHAAuthProtocol,
                                privProtocol=usmAesCfb128Protocol),
                                UdpTransportTarget((ip, 161)),
                                ContextData(),
                                ObjectType(ObjectIdentity(oid)),
                                lexicographicMode=False):
            if errorIndication:
                print(errorIndication)
                break
            elif errorStatus:
                print('%s at %s' % (errorStatus.prettyPrint(),
                                    errorIndex and varBinds[int(errorIndex) - 1][0] or '?'))
                break
            else:
                for varBind in varBinds:
                    results.append(' = '.join([x.prettyPrint() for x in varBind]))
        return results

they both get passed the same info OIDs and creds

Sync results:

['SNMPv2-SMI::mib-2.4.24.4.1.4.0.0.0.0.0.0.0.0.0.10.10.10.2 = 10.10.10.2', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.1.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.11.1.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.3.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.11.2.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.5.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.11.3.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.7.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.11.4.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.9.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1'...

async results:

['SNMPv2-SMI::mib-2.4.24.4.1.4.0.0.0.0.0.0.0.0.0.10.10.10.2 = 10.10.10.2']

which looks like the nextCmd isn't working possibly?

[etingof]

It seems that your async script misses a loop over nextCmd() while the sync version has a for-loop. Here is the example.

[ajwillo]

im not quite sure where I'm missing the loop?

ive tried what I thought was right below but it prints the first record many times instead of the next record

def snmp_walk_a(ip, oid, snmp_user, snmp_auth, snmp_priv):
    snmpEngine = SnmpEngine()
    while True:
        (errorIndication,
         errorStatus,
         errorIndex,
         varBindTable) = yield from nextCmd(
                            snmpEngine,
                            UsmUserData(snmp_user, snmp_auth, snmp_priv,
                            authProtocol=usmHMACSHAAuthProtocol,
                            privProtocol=usmAesCfb128Protocol),
                            UdpTransportTarget((ip, 161)),
                            ContextData(),
                            ObjectType(ObjectIdentity(oid)),
                            lexicographicMode=False
                        )

        if errorIndication:
            print(errorIndication)
            break
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBindRow in varBindTable:
                for varBind in varBindRow:
                    print(' = '.join([x.prettyPrint() for x in varBind]))

        varBinds = varBindTable[-1]
        if isEndOfMib(varBinds):
            break

[ajwillo]

Would anyone be able to assist me with this? is it a bug or an error in my code?

Thanks

[etingof]

This is still not quite clear what's wrong in your situation. Look, here is practically your script which works alright against public SNMP agent (you can just run this script locally):

import asyncio
from pysnmp.hlapi.v3arch.asyncio import *

@asyncio.coroutine
def run(varBinds):
    snmpEngine = SnmpEngine()
    while True:
        (errorIndication,
         errorStatus,
         errorIndex,
         varBindTable) = yield from nextCmd(
                            snmpEngine,
                            UsmUserData('usr-none-none'),
                            UdpTransportTarget(('demo.snmplabs.com', 161)),
                            ContextData(),
                            *varBinds,
                            lexicographicMode=False
                        )

        if errorIndication:
            print(errorIndication)
            break
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBindRow in varBindTable:
                for varBind in varBindRow:
                    print(' = '.join([x.prettyPrint() for x in varBind]))

        varBinds = varBindTable[-1]
        if isEndOfMib(varBinds):
            break

    snmpEngine.transportDispatcher.closeDispatcher()


loop = asyncio.get_event_loop()
loop.run_until_complete(
    run([ObjectType(ObjectIdentity('SNMPv2-MIB', 'sysDescr'))])
)

May be it has something to do with the rest of the code? Or with the agent you are querying?

I do not see anything suspicious in the function you've pasted.

WDYT?

[ajwillo]

ok, I copy and pasted your code exactly and then added my variables into create the below:

@asyncio.coroutine
def snmp_walk(ip, varBinds, snmp_user, snmp_auth, snmp_priv):
    results=[]
    print(ip)
    snmpEngine = SnmpEngine()
    i = 0
    while True:
        i = i+1
        print('Routes Found {}'.format(i))
        (errorIndication, errorStatus, errorIndex, varBindTable) = yield from nextCmd(
                            snmpEngine,
                            UsmUserData(
                                    snmp_user, 
                                    snmp_auth,
                                    snmp_priv,
                                    authProtocol=usmHMACSHAAuthProtocol,
                                    privProtocol=usmAesCfb128Protocol
                                ),
                            UdpTransportTarget((ip, 161)),
                            ContextData(),
                            *varBinds,
                            lexicographicMode=False
                        )

        if errorIndication:
            print(errorIndication)
            break
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBindRow in varBindTable:
                for varBind in varBindRow:
                    results.append(' = '.join([x.prettyPrint() for x in varBind]))

        varBinds = varBindTable[-1]
        if isEndOfMib(varBinds):
            break

    snmpEngine.transportDispatcher.closeDispatcher()
    return results

and its called with this

routing_table = await snmp_walk(device.poll_ip, [ObjectType(ObjectIdentity(device.routing_table_oid))], device.device.snmp_data.name, \
                                            device.device.snmp_data.auth, device.device.snmp_data.priv)

however I think I may have some error in there still, as there are 600ish routes in my table, however the script went crazy and was printing out "Routes Found 4483" before I quit it manually.

perhaps the while loop isn't breaking? are you able to advise where my issue may be?

Thanks

[etingof]

Ah, now the thing is that async API does not implement the lexicographicMode option. You may want to check out issue #233. In essence, here is how to break out of the SNMP GETNEXT loop:

@asyncio.coroutine
def run(varBinds):
    snmpEngine = SnmpEngine()
    initialVarBinds = varBinds
    while True:
        (errorIndication,
         errorStatus,
         errorIndex,
         varBindTable) = yield from nextCmd(
            snmpEngine,
            CommunityData('public'),
            UdpTransportTarget(('demo.snmplabs.com', 161)),
            ContextData(),
            *varBinds)

        if errorIndication:
            print(errorIndication)
            break
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBindRow in varBindTable:
                for idx, varBind in enumerate(varBindRow):
                    print(' = '.join([x.prettyPrint() for x in varBind]))

        varBinds = varBindTable[-1]
        if isEndOfMib(varBinds):
            break

        for varBind in varBinds:
            if initialVarBinds[0][idx].isPrefixOf(varBind[0]):
                break

        else:
            break

[ajwillo]

Thankyou for your help with this one, all is now working as desired