A little VB code that I wrote yesterday

[Whirlingdervish]

After realizing that I had no way to handle and record the information from my Q2 server logs without manually skimming them in notepad and grabbing out anything that I might need, I started putting together a little frontend program in VB6 and kinda surprised myself by remembering way more VB than I thought I did. (hadn't used it since vb3 or so in high school)

Most of the code is hacked together and inefficient on memory usage, and I broke some personal rules about hiding data from my functions that they don't really need to be able to access, but so far I managed to write a couple of subroutines in my spare time that read the log files, skim for player connects and store the player IP and the alias they used to a pair of arrays that are later combined with the existing player IP list (a comma seperated values file) that I plan on using as a searchable database.

So far I just wanted to prove that I could get that done, but I plan on making a few changes and implementing some code that will compare/grab the most recent date of connection, store the number of times a given alias/ip combo has been seen and possibly the client version info as well.

Here's a snippet:

Code: [Select]

' general declarations:

' define global arrays

Dim iplistarraya() As String
Dim iplistarrayi() As String
Dim logfilearraya() As String
Dim logfilearrayi() As String

' define global counters

Dim logfilelength As Long
Dim iplistlength As Long

' define global flags

Dim resetclicked As Integer
Dim ipfileopen As Integer
Dim logfileopen As Integer

' define global path variables

Dim logfilepath As String
Dim iplistpath As String

' ------------------------------------


Private Sub form_load()

' set initial values for flags, path vars, and counters upon startup

logfilepath = "c:\logs\"
iplistpath = "c:\logs\iplist.txt"
iplistlength = 0
logfilelength = 0
resetclicked = 0

End Sub



Private Sub logopenbutton_Click()

' using the path from the server log path global variable, open the ip list
' verify that server log file has contents (quickly error check path and file content)
' add contents of log file to log file array
' cull useless values from log file array and sort it
' make the log file buttons accessable (not greyed out)

Dim templogfilearray() As String
Dim temparraya() As String
Dim temparrayi() As String

Dim checkchar As String

Dim index As Long
Dim index2 As Long
Dim index3 As Long

index = 0
index2 = 0
index3 = 0

Dim aliasendpos As Long
Dim aliaslength As Long
Dim iplength As Long
Dim ipendpos As Long
Dim tempstring As String

aliasendpos = 0
aliaslength = 0
iplength = 0
ipendpos = 0

logfileopen = 0

If logfilepath = "c:\" Or logfilepath = "" Then
    MsgBox "The Path you have entered for the Log File is incomplete, Please Re-Enter."
Else
    ' open logfile for reading and if file is empty close it and ask for another path
    ' if it has contents, read contents into temp arrays and count # of lines
   
    indicator.Caption = "Opening File Please Wait.."
   
    Open logfilepath For Input As #1
   
    If LOF(1) > 0 Then
           
        While Not EOF(1)
           
            ReDim Preserve templogfilearray(0 To index) As String
            Line Input #1, templogfilearray(index)
           
            checkchar = Mid$(templogfilearray(index), 20, 1)
           
            If checkchar = "(" And InStr(templogfilearray(index), " connected from") <> 0 Then
                ReDim Preserve logfilearraya(0 To index) As String
                ReDim Preserve logfilearrayi(0 To index) As String
               
                ' calculate alias length and the start position
                aliasendpos = InStr(templogfilearray(index), " connected from") - 1
                aliaslength = aliasendpos - 20
                logfilearraya(index) = Mid$(templogfilearray(index), 21, aliaslength)
               
                ' calculate the ip length and start position
                ipendpos = Len(templogfilearray(index))
                iplength = ipendpos - aliaslength - 37
                logfilearrayi(index) = Mid$(templogfilearray(index), aliasendpos + 17, iplength)
               
                ' increment the value that will be the global logfilelength counter
                index = index + 1
               
            End If
           
        Wend
       
        logfilelength = index
       
        index = index - 1
       
        logfileopen = 1
        If ipfileopen = logfileopen And ipfileopen = 1 Then
            writebutton.Enabled = True
            Combinebutton.Enabled = True
        End If
        displaylogval.Enabled = True
       
        indicator.Caption = "Log File Loaded"
       
        ' call the sort function to arrange elements in arrays by case sensitive player alias
       
        QuickSort logfilearraya, 0, logfilelength - 1, logfilearrayi
       
        indicator.Caption = "Log File Sorted"
       
        ' call the appropriate cull function to cut out any duplicates from the arrays
       
        logcullfunc logfilearraya, logfilearrayi, logfilelength
       
        indicator.Caption = "Duplicate Log Values Removed"
       
        Close #1
    Else
        MsgBox "The Log File you have specified is empty, Please enter a different path, and retry."
       
        Close #1
    End If
End If

End Sub



Private Sub logcullfunc(aliasarray() As String, iparray() As String, arraylength As Long)

Dim temparraya() As String
Dim temparrayi() As String
Dim duplicates() As Long
Dim firstpos As Long
Dim index As Long
Dim index2 As Long
Dim tempcounter As Long

firstpos = 0
index = 0
index2 = 0
tempcounter = 0

ReDim duplicates(0 To arraylength) As Long
ReDim temparraya(0 To arraylength) As String
ReDim temparrayi(0 To arraylength) As String

ReDim Preserve aliasarray(0 To arraylength + 1) As String
ReDim Preserve iparray(0 To arraylength + 1) As String

While index < arraylength
    duplicates(index) = 0
    index = index + 1
Wend

' values in the arrays are pre-sorted by alias, but need to be also sorted by IP
' first scan through the alias array for how many duplicate instances there are for each alias
' store this duplicates value in a temparray using the same index position as the first alias
' in the array that is duplicated, so you have a start position to send the sort
' function for each different alias

index = 0

While index < arraylength
    ' if one alias matches the next alias count how many more there are and change the index
    ' position to skip them being counted on the next pass of the while loop
    If aliasarray(index) = aliasarray(index + 1) Then
        firstpos = index
        index2 = index + 1
        While aliasarray(firstpos) = aliasarray(index2)
            duplicates(firstpos) = duplicates(firstpos) + 1
            index2 = index2 + 1
        Wend
        index = index + duplicates(firstpos) + 1
       
    ' if they dont match, leave the value of the index in the duplicates array as a 0
    ' then increment the index so the loop proceeds to check the next entry as normal
    Else
        index = index + 1
    End If
Wend

' if the number of duplicates for an alias is not 0, send the sort function the arrays
' and the bounds to sort only that section which contains duplicates of the alias:
' the upper bound sent to the sort function will be the index position of the first entry
' added to the number of duplicates that follow it. the lower bound will be the index position.
' This will force the sort function to retain the sorting of the aliases from earlier
' and only work on sorting those entries by IP and grouping the duplicate values in
' consecutive blocks that the cull loop below will handle.

index = 0

While index < arraylength
    If duplicates(index) > 0 Then
       
        QuickSort iparray, index, duplicates(index) + index, aliasarray
   
    End If
    index = index + 1
Wend

' go through the array name by name, removing the consecutive duplicate IP entries

index = 0

While index < arraylength
   
    If aliasarray(index) = aliasarray(index + 1) Then
        If iparray(index) = iparray(index + 1) Then
            temparraya(tempcounter) = aliasarray(index)
            temparrayi(tempcounter) = iparray(index)
        Else
            temparraya(tempcounter) = aliasarray(index)
            temparrayi(tempcounter) = iparray(index)
            tempcounter = tempcounter + 1
        End If
           
    Else
        temparraya(tempcounter) = aliasarray(index)
        temparrayi(tempcounter) = iparray(index)
        tempcounter = tempcounter + 1
    End If
   
    index = index + 1

Wend

' refill the real arrays with the newly sorted temp arrays
' reset the global logfilelength variable to the new length value

index = 0
logfilelength = tempcounter

ReDim logfilearraya(0 To logfilelength) As String
ReDim logfilearrayi(0 To logfilelength) As String

While index < tempcounter
    logfilearraya(index) = temparraya(index)
    logfilearrayi(index) = temparrayi(index)
    index = index + 1
Wend

End Sub




Private Sub QuickSort(searcharray() As String, intBottom As Long, intTop As Long, slavearray() As String)

Dim strPivot As String
Dim searcharraytemp As String
Dim slavearraytemp As String

Dim intBottomTemp As Long
Dim intTopTemp As Long

intBottomTemp = intBottom
intTopTemp = intTop

strPivot = searcharray((intBottom + intTop) \ 2)

While (intBottomTemp <= intTopTemp)

'comparison of the values is a descending sort using the bounds that were passed to the function
'and cutting the list in half based on them

    While (searcharray(intBottomTemp) < strPivot And intBottomTemp < intTop)
        intBottomTemp = intBottomTemp + 1
    Wend

    While (strPivot < searcharray(intTopTemp) And intTopTemp > intBottom)
        intTopTemp = intTopTemp - 1
    Wend
    If intBottomTemp < intTopTemp Then
        ' the slave array element is sorted based only on the values from the
        ' searcharray comparison, in order to keep the value in the same index
        ' position as that of the corresponding value in the other array
       
        slavearraytemp = slavearray(intBottomTemp)
        slavearray(intBottomTemp) = slavearray(intTopTemp)
        slavearray(intTopTemp) = slavearraytemp
   
        searcharraytemp = searcharray(intBottomTemp)
        searcharray(intBottomTemp) = searcharray(intTopTemp)
        searcharray(intTopTemp) = searcharraytemp
    End If

    If intBottomTemp <= intTopTemp Then
        intBottomTemp = intBottomTemp + 1
        intTopTemp = intTopTemp - 1
    End If

Wend

'the function calls itself recursively until everything is correctly sorted

If (intBottom < intTopTemp) Then QuickSort searcharray, intBottom, intTopTemp, slavearray
If (intBottomTemp < intTop) Then QuickSort searcharray, intBottomTemp, intTop, slavearray

End Sub



It's nothing fancy, and has next to no error handling so far, but in combination with a few other subroutines and such, it has managed to effectively read and skim out the player alias and ip values that I wanted straight from the server logs, sorted them, culled out duplicate entries and written this info to a file.

 

[quadz]

Cool, man.

Extra props for actually accomplishing it in VB6.   I inherited a VB6 project many years ago, and it was amazing to me how complicated simple problems can become in that language, which in theory was supposed to be high-level.

Here's similar code in ruby to parse an OSP logfile, and output tab-separated unique IPs, playernames.

Code: [Select]

def parse_name_and_ip(line)
  # OSP logfile connects look like:
  # [2009-03-02 15:42] (asdf connected from 123.45.67.89)
  if line =~ /\((.+) connected from ([\d.]+)\)/
    [$1, $2]
  else
    nil
  end
end

def parse_connects(file)
  connects = []
  file.each_line do |line|
    name, ip = parse_name_and_ip(line)
    if name
      connects << [ip, name]
    end
  end
  connects 
end

# print tab-separated unique connects sorted by IP
connects = parse_connects(ARGF)
uniq_connects = connects.uniq   # :D
sorted_connects = uniq_connects.sort   # :D

sorted_connects.each {|ip,name| puts "#{ip}\t#{name}"}


For fun, here's a one-liner version of the above:

Code: [Select]

ARGF.read.scan(/\((.+) connected from ([\d.]+)\)/).uniq.sort_by{|name,ip| ip}.each{|name,ip| puts "#{ip}\t#{name}"}


Regards,

[Whirlingdervish]

 

That is so much more efficient that it's scary!

I know what you mean about VB6 too.. It seems like it makes simple tasks into monumental undertakings.
If I had used the relatively similar VB .Net instead of the old vb6, I could have saved a lot of time and trouble that I spent reinventing the wheel.

I laughed when I realized I was writing code that was later included into the Basic lexicon as functions to deal with arrays without a gazillion loops and conditionals every time you want to delete an element out of the middle of it and scootch everything down a place or two..

I kept thinking to myself, If I'd used something like C++ or Java I could have cranked out an uglier frontend that worked a hundred times faster and did everything I wanted it to already!


I'm a sucker for that pretty windows GUI.

 

[reaper]

nice

for database stuff in scripts you might find this handy http://www.sqlite.org/ , you can pipe in commands to import comma delimted files with the built in import command.  like echo ".import cvsfile" | sqlite3 databasename , or use whatever type of  database connection you want, but you'd have to do the import in SQL. 

[Whirlingdervish]

gave that sql library a little look over..

pretty slick stuff. looks very nicely maintained and super efficient.

 

[reaper]

gave that sql library a little look over..

pretty slick stuff. looks very nicely maintained and super efficient.

It can do a lot pretty easily.  I just delete the database on the fly and recreate it.  Then I don't have to worry about array sorting, and have a relational database to mess with.  I'm sure the better way is to import the CVS file with SQl and the database driver, but as you can see I don't care.  It's so easy to have a database now.




unlink "/var/scripts/serverinfo.db";

#we will create the tables with the db driver
$linkid = sqlite3_open( '/var/scripts/serverinfo.db')
   or die sqlite3_error();
$stmtid = sqlite3_prepare('create table portlist(port,vlan)')
   or die sqlite3_error();
$resid = sqlite3_execute ($stmtid)
   or die sqlite3_error();

#create next table which contains the below columns

$stmtid = sqlite3_prepare('create table vlanlist(ip,mac,vlan)')
   or die sqlite3_error();
$resid = sqlite3_execute ($stmtid)
   or die sqlite3_error();

#create next table which contains mac address and port

$stmtid = sqlite3_prepare('create table maclist(mac,port)')
   or die sqlite3_error();
$resid = sqlite3_execute ($stmtid)
   or die sqlite3_error();



#a bash script is called to import the data from cvs files into the database tables, since separator and import need to be called in the same sqlite3 session, we will pipe the input as a file into sqlite3, as far as I can tell the perl sqlite3 driver cannot use the builtin sqlite3 functions

system("/var/scripts/import.sh > /dev/null 2>&1");

#prepare statement, join on vlan column in portlist table and vlanlist table
$stmtid = sqlite3_prepare( 'select * from portlist inner join maclist on portlist.port = maclist.port outer join vlanlist on vlanlist.mac=maclist.mac' )
      or die sqlite3_error();
   
#execute statement and store the result
$resid = sqlite3_execute( $stmtid )
      or die sqlite3_error();
 
#print only one entry per vlan
  while( @row = sqlite3_fetch_row( $resid ) )
{
      print join( ', ', @row ), "\n";
}   


/* import.sh
echo -e ".separator ,\n.import /var/log/vlanslist portlist\n" | sqlite3 /var/scripts/serverinfo.db
echo -e ".separator ,\n.import /var/log/arpandmac vlanlist\n" | sqlite3 /var/scripts/serverinfo.db
echo -e ".separator ,\n.import /var/log/macandport maclist\n" | sqlite3 /var/scripts/serverinfo.db

*/

[Whirlingdervish]

After a bit of dickin around, I managed to finish up the first build of this little utility and it seems to work well so far.
It will parse the OSP logs, display the values it grabbed, and it can combine this information with the stored list and write it to file for later use.
The stored values go in a file that works as an IP list, and you can search them by IP, subnet range, alias, or even a part of the alias.

The only hiccup I've noticed is that linux servers use an endline character that vb6 is unable to handle correctly, so I have to open the logs in wordpad and save them prior to scanning. Hopefully I'll figure out a way to fix that for future builds.

here's the first release:
http://ir.tastyspleen.net/files/serverlogutil.zip


and a screenie:

[reaper]

"
Systems based on ASCII or a compatible character set use either LF (Line feed, 0x0A) or CR (Carriage Return, 0x0D) individually, or CR followed by LF (CR+LF, 0x0D 0x0A); see below for the historical reason for the CR+LF convention. These characters are based on printer commands: The line feed indicated that one line of paper should feed out of the printer, and a carriage return indicated that the printer carriage should return to the beginning of the current line.
"

The text files created on linux end new lines with 0x0A (referenced by /n), while windows text files will end new lines with 0x0D,0x0A (referenced by /r/n, in C etc).  So you can use something like the tr utility, which is also a function in VB: http://en.wikipedia.org/wiki/Tr_(Unix)

I think it's interesting programs like telnet will convert "/n" to "/r/n", which is supposedly the presentation layer of the OSI model, but it seems there's been a lot of confusion about that.


http://linuxcommand.org/man_pages/dos2unix1.html  dos2unix unix2dos also does it, but i'm pretty sure VB has a TR like function.  So the TR function will basically go through the text and convert "\n" to "\r\n"




I didn't think you could get another players IP?
It'd be cool to put in a player name and get references to the chat with them in the log, not sure if it does that, or it's something you wanna do.

[Whirlingdervish]

I didn't think you could get another players IP?
It'd be cool to put in a player name and get references to the chat with them in the log, not sure if it does that, or it's something you wanna do.

So far, it just takes and stores the player name and corresponding IP for that instance that they connected out of the server logs that OSP tourney makes on the server's webspace when the servers are running. If I didn't admin a server, I'd be equally unable to see anyone's IP, but in this case, if you connect to IRTDM or IRTDM2 you're gettin logged and when it comes down to it, I'm the only one who pokes through those logs.

as they are, the logs consist of many megabytes of line after line of the server chat, rcon commands being sent, and connection instances. You could open one in wordpad or another competent text editor, and use the find function to search for a certain IP or a certain player name to scroll through the log for every time that they've said something or connected, but this can be very time consuming and is generally only worth it if someone did something to stand out and get the attention of the admin.

this util is pretty much just a way for me to easily parse every unique IP and alias that connects to the servers over the course of time. the ip list that it generates will be used when I have to craft a ban or force anticheat on someone so that I can find every IP and subnet mask they've used and make the sure they cant get around it.

some side effects of the way I made the search by IP work, is that I can search to see who (of the people who've ever played on the servers) will be directly affected if I force AC on a certain subnet, and who has a very similar IP to the next guy (occasionally this helps spot someone whos bant aliasing and trying to get around the ban with a change of IP)

features that I will probably add will include storage of the last date of connection for every unique IP, and the different q2 clients and versions that they have tried to connect with. I dont rly care what people say in chat and I keep the logs just in case, so storing that is a non-issue for me in later builds.


That little blurb on linefeed chars above will help with my other little problem tho. I've considered searching the log first and replacing chars using binary to locate them instead of trying to handle the characters in strings, but hex may just be easier than both.

[reaper]

I gotcha now, I figured you must of been the server admin.

For the linux files, all you have to do is convert \n (0x0A) with \r\n (0x0D 0x0A).  The utility unix2dos will do it.  Like "unix2dos filename" adds the LF character (0x0D). 

In *nix you could do something like "sed 's/$/\r\n/' filename"  , I think in VB you probably would use regular expressions, or you could look at the file and change 0x0A to 0x0D,0x0A, i'm not sure how the offset works in that, but I guess it's ok.

[QwazyWabbit]

The only hiccup I've noticed is that linux servers use an endline character that vb6 is unable to handle correctly, so I have to open the logs in wordpad and save them prior to scanning. Hopefully I'll figure out a way to fix that for future builds.

Linux uses LF characters (0x0a) as end of line, Windows uses CR-LF (0x0d, 0x0a) end of lines. If you FTP the logs from the server to the Windows machine as "ascii" files (type ascii in the ftp mode) it will translate the EOL's for you. (Linux did this (IIRC) to conserve disk space, translating it to cr-lf to the terminal device.

Or using VB6, you can translate your Linux text files by using VB's constants: vbLf and vbCrLf as it reads the lines in. The trick is to detect when you are seeing them and choosing the correct translation mode. Maybe grabbing the first line and setting a boolean if you see a vbLf at the end of it might work. Then pass the log thru the translator first.

You may also have to beware Unicode output from VB.