Stats

  • Locked Locked
  • Replies 16
  • Subscribers 143
  • Views 18857
  • Members are here 0
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

What is the most efficient approach to record each line from a input file ?

Charley Chen

 Hi All,

I use table to record each line & its value , but it will become slower and slower when time left .

Though I can get table[A11100] = list(1000000 2000000 3000000 4000000) very quickly , But must when it finished loop.

What is the best way to do it ?

;write a template file
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 
        getCurrentTime()
        outPort = outfile("test")
        count = 1000000
        fileCount = 1
        for(i 1 count
            fprintf(outPort "1000000 2000000 3000000 4000000\n")
        );   
        close(outPort)
        getCurrentTime()
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 

        ;read each line to record
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 
        nextLine = nil
        inPort = infile("test")
       getCurrentTime()
 when(inPort
      fileCount = 1
      table = makeTable("table" nil)
      while(gets(nextLine inPort)
              qq = parseString(nextLine "\n")
              str = sprintf(nil "A%d" fileCount)
              table[str] = list(1000000 2000000 3000000 4000000)
              fileCount++
      );while   
 );when
      inPort = nil
      getCurrentTime()
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

 

Thank you,

Charley

Parents
  • Andrew Beckett

    Charley,

    Calling needNCell() multiple times is not a problem - it does not add that number, but sets the upper limit. This means you cannot reduce it once allocated, because  you've pre-allocated a pool ready to be used by SKILL. However, SKILL will be able to reuse any of the list or string cells which are no longer in use (for list cells or strings respectively) - this is what garbage collection is all about. For example, before I run your example reading 1 million lines into a table, if I run gcsummary() :

    -----------------------------------------------------------
    Type       Size   Allocated     Free      Static   GC count
    -----------------------------------------------------------
    ...
    list         12     1593344    26124     1191936          3
    ...
    string        8      167936    81680      229376          0

    After running it, I see:

    -----------------------------------------------------------
    Type       Size   Allocated     Free      Static   GC count
    -----------------------------------------------------------
    ...
    list         12    50089984   719256     1191936          6
    ...
    string        8     8966144   818144      229376          3

    So if once you're done you do:

    table=nil
    gc() ; force a garbage collection rather than waiting until the system thinks one is necessary
    gcsummary()

    -----------------------------------------------------------
    Type       Size   Allocated     Free      Static   GC count
    -----------------------------------------------------------
    ...
    list         12    50089984 48842892     1191936          7
    ...
    string        8     8966144  8859424      229376          3

    So what you're seeing there is now it has freed up lots of list and string slots - which can be reused by other programs within the same session.

    Hope that helps!

    Regards,

    Andrew.

Reply
  • Andrew Beckett

    Charley,

    Calling needNCell() multiple times is not a problem - it does not add that number, but sets the upper limit. This means you cannot reduce it once allocated, because  you've pre-allocated a pool ready to be used by SKILL. However, SKILL will be able to reuse any of the list or string cells which are no longer in use (for list cells or strings respectively) - this is what garbage collection is all about. For example, before I run your example reading 1 million lines into a table, if I run gcsummary() :

    -----------------------------------------------------------
    Type       Size   Allocated     Free      Static   GC count
    -----------------------------------------------------------
    ...
    list         12     1593344    26124     1191936          3
    ...
    string        8      167936    81680      229376          0

    After running it, I see:

    -----------------------------------------------------------
    Type       Size   Allocated     Free      Static   GC count
    -----------------------------------------------------------
    ...
    list         12    50089984   719256     1191936          6
    ...
    string        8     8966144   818144      229376          3

    So if once you're done you do:

    table=nil
    gc() ; force a garbage collection rather than waiting until the system thinks one is necessary
    gcsummary()

    -----------------------------------------------------------
    Type       Size   Allocated     Free      Static   GC count
    -----------------------------------------------------------
    ...
    list         12    50089984 48842892     1191936          7
    ...
    string        8     8966144  8859424      229376          3

    So what you're seeing there is now it has freed up lots of list and string slots - which can be reused by other programs within the same session.

    Hope that helps!

    Regards,

    Andrew.

Children
No Data

Community Guidelines

The Cadence Design Communities support Cadence users and technologists interacting to exchange ideas, news, technical information, and best practices to solve problems and get the most from Cadence technology. The community is open to everyone, and to provide the most value, we require participants to follow our Community Guidelines that facilitate a quality exchange of ideas and information. By accessing, contributing, using or downloading any materials from the site, you agree to be bound by the full Community Guidelines.