Delete shapes under specific layers not working consistently

My guess is that you have some long lists and using apply('nconc ovlp) etc might be having some trouble (applying a function with a very large number of arguments in the order of 60000 or so can cause trouble).

Perhaps try:

ovlps = foreach(mapcan shp int
    dbProduceOverlap(cv shp~>bBox 0 deleteLayer)
    )

ovlps2 = foreach(mapcan shp int
    dbProduceOverlap(cv shp~>bBox 0 list("text" "drawing"))
    )

instead. This will directly give you the concatenated lists.

Andrew

Thanks Andrew, the issue was the foreach loop itself. Modified it as following and it works now.

ovlp1 = nil

foreach( shp int

    ovlp = dbProduceOverlap(cv shp~>bBox 0 deleteLayer)

    ovlp1 = append(ovlp1 ovlp)

    )

SSR said:

Thanks Andrew, the issue was the foreach loop itself. Modified it as following and it works now.

This makes no sense - there's nothing wrong with the foreach loop. Your new implementation is potentially rather inefficient because it's using append - which will add to the end of the list, and is slow for large lists. Given that you are trying to build a flattened list, you might be better off swapping the order of the arguments to append, as ovlp will (in general) be the shorter of the two lists and so is quicker to reach the end. It would also be better to use nconc. Or better to use foreach mapcan!

Given that the original error didn't make sense to me either - I wouldn't have expected the apply('nconc) to fail with a value stack issue - my guess is that the problem is actually elsewhere in your code and this part is just the piece that is finally hitting the value stack limit. If you use foreach with mapcar or mapcan, it actually transforms it from this:

ovlps = foreach(mapcan shp int
  dbProduceOverlap(cv shp~>bBox 0 deleteLayer)
)

into this:

ovlps = mapcan(
  lambda(shp) dbProduceOverlap(cv shp~>bBox 0 deleteLayer)
  int
)

In other words, it's introducing a new (anonymous) function, and then calling that repeatedly for each element in the list. It might be this extra function that's pushing it over the limit, but that must be because of something going on above in the code (outside of what you've shared). That's the problem that needs fixing, not this - you are merely papering over the cracks otherwise.

Andrew

Andrew, appreciate the detailed reply, I've tried using the method that you have mentioned and it seems to work, but there is an issue. 

As per my usage, the script is supposed to delete all the dummy layers that are touching/overlapped with hilite/drawing9 layer and the size of the layout is about 14000x3500um. It seems like some of the dummy layers are removed and the rest are not. If I were to call the function again, it deletes some more, but not all. I'm not sure what's the limitation here.

Restarting the virtuoso fixed this issue, i'm not sure how though...