Esdee

linux threads and forking (and zeroc ice)

2011-09-25T13:27:00.000-07:00

so we have this nice icegrid setup with python nodes tied with sip to underlying c++ code. on new rpc call, ice creates a new thread to handle it, but as the underlying c++ code is not threadsafe we fork from this thread and continue execution in a new process.

everything works perfectly, but one day we decide to remove the python layer and go with c++ all the way

and then something weird started happening - forked processes started hanging, crashing, a total mess...after some research we found out several articles mentioning that it is a bad idea to mix linux threads and forks because of possible copied locked mutexes in the child process and this is exactly what we had observed.

so how we solve this? and why the python+cpp solution worked fine?

as a second solution we modified our code to use a thin python wrapper again over the main c++ functionality in attempt to copy the behavior from the original solution, but again the forked child processes had copied locked mutexes that caused them to hang

so, how come? in the original solution A the underlying code is pretty much the same as B and C, what appears is that some ice threads are calling localtime_r in the moment when we fork and when the forked code tries to execure localtime_r it locks.
but why does it not happen in the original python/cpp solution? why does it happen in the new python/cpp solution? are there some python flags to help avoid this? the original code would spend some time jumping from python to cpp, while the new code would only enter cpp code once and return the results.

in any case it seems a mess so we went for another completely different solution, but the headache was/is huge

c++: references vs. pointers

2011-06-21T15:32:00.001-07:00

there is some campaign at my company right now. its about replacing/using references in new code as opposed to pointers because this is ....modern; right.

i mean, i understand all the benefits of using shared pointers, smart pointers, boost utils, but please do not do it on mandatory basis; don't insist it is applied in every case and in every line of code, using references in the hands of an inexperienced programmer can raise even bigger problems than a bare naked 'C' pointer memory leak or such.

two very common bugs i've encountered after reviewing some code that followed the new 'guidelines':

A& func1()
{
A* a = new A;
return *a;
}

don't you think this is ugly in the least? not to mention, that nobody knows where 'a' is dealocated, if ever

another example (this one is sweet as honey):

B& func2(const B& bIn)
{
B b = bIn;
return b;
}

it took me quite a few moments actually to realize that this method is returning a reference to a local object. no wonder then that the all of the sudden the returned object behaved strangely and had inconsistent data, as the author developer complained

fun, fun, fun...

anyway...all is good, references are fine and useful, but please don't say that using pointers is old fashioned, don't want to get started again...

:D

good night!

using apache qpid with persistence

2011-04-30T13:02:00.000-07:00

that's just a quick post about two weeks struggle with a problem which was eventually solved in half an hour.

as at my company we (i) decided to use apache QPID as a message queue framework, we certainly needed to use persistance for a queues and messages. building qpid itself was really easy and straight forward, and for building the persistance module msgstore.so i was following the directions posted at Lahiru Gunathilake's Blog here. i build the whole setup on my local machine and everything was working just perfectly - messages were sent persitently, queues were durable etc.

then i started deploying on the dev servers

the qpid broker started crashing on startup.
the qpid broker started crashing on startup.
the qpid broker started crashing on startup.

and on and on and on

it was easy to detect that the problem appeared when the broker was started with --load-module=msgstore.so and that it seg faulted when attempting to create a berkeley db database. but why??
well...on my local machine i am running fedora, and the dev server rules with CentOs.
my fedora has berkeley db 4.8 and the server has 4.5. this was the first 'ding'.

now, how to trick the qpid configure into using db4.8 instead of db4.5? we couldn't just install the new libs as there might have been incompatibilities with the product, so what to do? we tried modifying configure scripts, playing with sym links etc - i myself don't have much experience with linux so was relying mostly on the admin guy, but he was helpless with this too - seg fault after segfaul, while on my machine the broker hummed silently, transfering persistant messages and maintaining durable queues.

then after a loooot of reading we came to an obvious solution - the qpid configure itself gave it too us, and hadn't i been too shortsighted, this would have been done days ago -

just build berkeley 4.8 with --perfix to place it in a separate directory (say $BDB48), away from the db4.5 libs the product needs, and then, before running qpid configure, run these

export CPP_PATH=$BDB48/include
export LIB_PATH=$BDB48/libs

(i'm writing these by memory, check qpid's configure script with --help to see the correct ones)

after that run configure and it will find and use the berkeley libs you wanted it too!

awesome!
now lets do the same in production where we have RedHad 5. BOOM! here goes the so familiar seg fault that we managed to escape by the clever export trick. why? what is so different on redhat???
another week of research, testing, breaking, hair-tearing followed, this time with no result.
i was even contemplating to run a virtual fedora machine on the redhat server, just and so to have the familiar setup and start the broker with persitancy there.

luckily i didn't have to -
one day i sat with the it manager to explain him the situation and while we were scratching our heads we looked at persistance module's readme.txt where it said that it was tested with berkeley db 4.3.
so what? we have db4.8 and it should be fine, right?
what if we gave it another try, but not with the 4.8, but 4.3 set-up the same way?
this took about10 minutes to setup and when i hit 'enter' for qpidd --load-module=msgstore.so, before my eyes was the beautiful log dump, saying that the module was loaded and so on and so on ....

aaah....rtfm? the thing is we actually tested db4.3 when facing the initial problem of setting up the dev servers. this failed somehow, and so we didn't consider this when fighting with production setup

anyways - this post grew as long as the first two episodes of "game of thrones" that i watched today :D

have a good night and don't give up the fight!

giving back

2011-04-30T12:42:00.000-07:00

i haven't been posting new articles, and even logging-in to this blog for probably years, but today i sat with the idea to share some more recent experience and noticed there were comments and "thanks", waiting to be approved (some sitting in for more than a year)...useless to say it warmed my heart that something i posted with the idea to be a dev diary read only by my self, actually turned out to be helpful for someone else out there, and that i incidentally have been giving my share to the greatest invention of our era - the Internet, the social network, the virtual community of knowledge and common interests, "The Void Which Binds"

alright now - on to the next problem :D

OSSP mm - Shared Memory Allocation

2009-10-30T17:16:00.000-07:00

i played a few days with the mm shared memory lib. i must say i liked it!
it is easy to understand and easy to implement into existing code.
of course the lack of new()/delete() is somewhat limiting, but not that difficult to overcome in most cases.
alternatively, on our project we were looking into boost:interprocess, but i was kinda reluctant to use it, mostly because of the syntax (btw i found a boost/ace alternative at POCO, the users and designers say it provides boost/ace-a-like functionality, even if it cannot compare in perfomance, but the best gain is the C#-oriented syntax; probably worth it to examine).

there was one really weird problem with mm though, when i got the impression that mm_lock() wasnt working.
the workflow was a follows:
we have a hash located on the shared memory segment and before i get() from it i mm_lock it and subsequently mm_unlock() it. still i noticed that many times same value was being written to the hash, which was unexpected by design.
i fired our test up to create a few working processes and breaked in each one before a get(). the locks seemed to work, but after a put() suddenly the lock seemed to vanish in the haze?! digging deeper i found out that in put() we of course try to allocad some new memory in the shared mem segment and traceing down into mm_malloc i found out it was trying to lock too, and when done - calling mm_unlock()! piece of bad design or weird side-effect? either way, apart from that my impressions with OSSP mm were great. however it turned out that the boost guys go a few more tricks up in their sleeves, especially the name persisted storage seemed a very attractive idea for our purposes and we eventually decided to use the boost approach.
so...moving on and up again :)

random performace thoughts

2009-10-19T12:08:00.000-07:00

just wanted to lay things out so i can see them more clearly. if it doesn't make sense to you, dear readers, please excuse me :)

so, we have many hashes (maps, containing cached data), but due to our implementation with fork() all the data is lost once a particular task is done. i had the idea of storing all those in a berkeley db so that every new forked task gets it from there and doesn't have to fill the hash by itself, but then - what is a task gonna read from the db when it doesn't know what it is gonna need beforehand. we cannot read everything as the data is HUGE and we cant read bit by bit as this slows down performance a lot - we had a test case like this: we tried using memcached as a remote cache, but this slowed us down even if we read from it only once into a local hash and all next calls where local (quick); if we hadn't read into local hash first, the slowdown would be even more devastating.

one possible solution is for the main process to reread all from the stored data and all children would get it via fork/copy-on-write and then put any changes/additions back into the db for the parent to read when a new task requests start.

i have another one around - when if we realise what we need from the cache much earlier than when we need it? we could then start a thread to pre-load the expected data while the main thread goes on processing and when it reached the point when the hash is needed - voila! its all up there! in case there are gaps they can be filled runtime, should be a small percentage of the overall effort

i'm working on a shared memory solution, but all these got me into doubt about the performance hit, similar to hitting the memcached everytime (by the way, i found out about OSSP mm shared memory allocation project at http://www.ossp.org/pkg/lib/mm/, and it looks really good. i dont know how i missed it earlier, guess i was looking for a std-kind-of allocator and got ptu back by some messages that boost had given up on the idea. another btw - is the new std with C++x0 gonna include something like this? from what i read it does include some fancy new stuff related to multithreaded and multicore programming which is awesome. would be great once we get our hands on it)

so where was i....aaah...back to the drawing board, time for another brainstoriming session...tomorrow :D

More on Berkeley DB

2009-02-11T16:05:00.000-08:00

alright! time passes on and one gets wiser (or feels sillier and has more questions to ask).

the project i'm working on is written in python with c++ interop throught sip.
at one point we had to move from the solely single-threaded design (who would design a single-threaded SERVER may i ask, duh) to a scalable architecture.

the first obstacle we hit was the mighty GIL of Python (oh my, what didn't i try to overcome this...) - fighting this monster made me an expert in providing solutions for scalability and performance optimizations. or not.

anyway, one of the problems we had was that somehow the data we read from berkeley db would become corrupted seemingly without any reason. dude...what a mess. our system is complicated enough so it took some time till we came to investigate the possibility that bdb was somehow the reason for this.
i studied the solutions for multithreaded and multiprocess access to bdb, added DbEnv, also played a bit with DB_DBT_USERMEM because we thought that simply relying on DB_DBT_MALLOC might lead to mem leaks (i still have to thouringly test and make sure that this is not an issue).
eventually we came to this - whenever you fork and the parent and any children processes share bdb handles - CLOSE AND REOPEN THE DB HANDLES IN THE CHILD PROCESS!
this can save you a couple of hours (or weeks) of hitting your head against the wall, so do this, get your bonus and go happilly home to your family for the day

cheers

to be continued...

getting to know Berkeley DB

2008-11-11T13:54:00.000-08:00

getting to know Berkeley DB came upon this link:

http://simonwillison.net/2003/Nov/26/discoveringBerkeleyDB/

and this comment:

My only complaint about BerkeleyDB is that it's a wee bit flaky when not used with transactions. Databases can become corrupted, processes can deadlock, etc. I've found problems even when using CDS mode. Only problem with wrapping everything in transactions is the performance hit. So, here's what we came up with as a compromise (we use the excellent BerkeleyDB.pm module from our Perl code): lock the entire database on write with a semaphore. The overhead is negligible in terms of speed, but it's done a remarkable job of keeping our indexes very clean!

lock the entire database on write with a semaphore
???
say again?

Update (a few hours later)
i'm beginning to understand what the comment author meant....my rdbms experiece will freak out with these new ideas in berkeley db...

rpmdb: PANIC: fatal region error detected; run recovery

2008-10-17T12:13:00.000-07:00

i started playing with fedora linux and today got this error while unstalling a package...
rpmdb: PANIC: fatal region error detected; run recovery
or actually many of these msgs

so this one helped to quickly resolve this

http://www.redhat.com/archives/fedora-list/2006-October/msg03072.html

thanks!

wireless configuration: Incorrect MFC DLL version installed

2008-09-17T13:43:00.000-07:00

several hours came and went banging my head against the wall after getting this error msg.
all started when i decided to finally uninstall vs 2003 and i guess it removed some dlls that i shouldn't have to...
i'm fairly pissed off right now so lets skip to the solution:
after deciding not reinstall windows because of this, at least i decided to put xp sp3, although i didn't really needed to...anyway this didn't help =/
after absurdly many hours of google search i came to this page - http://www.techsupportforum.com/microsoft-support/windows-xp-support/169616-incorrect-mfc-dll-version-installed-system.html
and this hint led me to open my services tab, kill the fukking 'tray service' and write this blog, quite sardonically if i may add...
damn software :)

on the market

2008-09-15T07:46:00.000-07:00

ok, so first interview came and went - what conclusions can be made?
the interviewer asked me about App domains
I was never really interested in those, said one of the benefits is the version control, gotta check this.
On multithreading...should i read more about some test cases? i spoke of some of my experience with it, its funny that when you have to talk about it its sometimes hard to elaborate on the subject...
WCF, now thats a weak point - i have experience with .Net remoting, but so many years ago I could hardly say any details, but it felt bad when the guy asked me had I had even read about WCF and I said "err...no" - gotta take care of this too, i guess
he also asked me if i had any experience with MS SQL Server, but when i started talking and mentioned the two certifications i have in this he seemed satisfied enough.
hmm...what else was there...

oh but of course - Web services - the last positions had me implementing solutions reading/writing from Web services, and this was good as when he asked me about securing web services i was stuck until i remembered how one of the service providers worked - by supplying a cookie to be used in each request after the initial login/pwd shakeup

Class member Entity.Id is unmapped.

2008-08-05T07:58:00.000-07:00

ok, the problem come from the fact that we have a number of tables (linq entities), but only a few of them have the Id column, and only on those few i need to run a certain method.
so how to do this?

i created the generic method

MMethod(int id) where T : Entity, IEntity
{
var r = from s in source.GetEntity()
where s.Id == id
select s;
}

and here i started getting the errors...
first of course i could not use s.Id because the general T doesn't have Id member defined.
so i started looking where to define it - and first defined it in the interface.
now, the interface would have the property Id and some of the linq entities would also have it, but on evaluation of the var r i would get

The mapping of interface member [...] is not supported

nice...
some reading on google and i started looking for another place to add this member.
the other obvious choice - the entity class.
this seemed to be my last chance, after i tried coming up with an extension method or whatever desperate solution i could think of.

so, member added, all compiled nicely, but while in debug i decided to see what the var r evaluates to and saw a sweet message:

Class member Id is unmapped

so...WTF? Id is now nicely defined and visible everywhere, so how come this error? and what does it mean??

some browsing in google, and i come to this page

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2446465&SiteID=1

at first quick look i couldn't find any solution, but, ooo how nice that i didn't close the page immediately, because after some time i went back to it, as something stuck as a neat idea - the guy was talking about a possible bug in linq implementation, but gave a few examples of when a statement worked and when it didn't, so i decided to give a try to one of the statements that he claimed to have worked and....voila!
problem resolved.
while waiting for the build i came upon another page that confirmed this solution here: http://stuff.hornbostel.com/?cat=10

neat
thanks guys

xlLocalSessionChanges not working

2008-07-29T07:29:00.000-07:00

"xlLocalSessionChanges not working"

this is the conclusion i reached when i passed XlSaveConflictResolution.xlLocalSessionChanges to workbook.SaveAs and the maddening alert showed up again.
before that i wasted a few hours wondering why my server application seemed to hang at the step when it should be writing some data to excel/csv file.
i was watching the results remotely through rdp and only later it occured to me that while i was waiting, on the server an alert appeared requiring user input - simple Yes/No/Cancel which I wasn't able to see through my remote session....

now this - setting the value to always overwrite the file and still this alertbox?

good think i quickly came upon this page:

http://www.notes411.com/dominosource/tips.nsf/0/B4EC9AD3C6CC8398802571790046F9DB!opendocument

turns out the ExcelApplication has a property that has to be set as well so this all works...cracks....excel.DisplayAlerts = false

Thanks, Mr. SCRobinson that tip had been very helpful...

The request was aborted: A connection that was expected to be kept alive was closed by the server

2008-07-25T13:00:00.000-07:00

it all started with this

The operation has timed out
at System.Net.HttpWebRequest.GetRequestStream()

first i thought this might be because of firewall, proxy, something local for our network...then we turned our attention to the counterparty - they chased their logs and couldn't find anything related to timeouts or connection drops for our endpoints...
then crazy testing began - the crazy thing was that all seemed to work on the local machined and in production, then it stopped working in production, and later it stopped working in dev also...nothing miraculous here - turned our we tested wrongly in dev, so all failed everywhere.
this code is a part of a nightly process so it quickly became a pain to wait and see if the nightly process worked and if not (most of the times) run the processes manually..pain pain pain...

so before this weekend i finally sat and decided to dedicate all my time to solving this fundamental problem

i had already set KeepAlive to false and this 'seemed' to work for some time...before coming back with fierce force

so the long chase on the net began...

the first step in the correct direction was to add .Net network tracing to the app, through this app.config setting:

<system.diagnostics>
<sources>
<source name="System.Net" tracemode="includehex" maxdatasize="1024">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Sockets">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Cache">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
</sources>
<switches>
<add name="System.Net" value="Verbose"/>
<add name="System.Net.Sockets" value="Verbose"/>
<add name="System.Net.Cache" value="Verbose"/>
</switches>
<sharedListeners>
<add name="System.Net"
type="System.Diagnostics.TextWriterTraceListener"
initializeData="network.log"/>
</sharedListeners>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="file" type="System.Diagnostics.TextWriterTraceListener" initializeData="networktrace.log"/>
</listeners>
</trace>
</system.diagnostics>

this pinpointed the problem and helped develop a reproduce-case: the time-out reoccured every third time HttpWebRequest.GetRequestStream() was called - awesome, so now what?

well the trace at least gave me hints no to what too look for in google - the timeout occured after these lines

System.Net Information: 0 : [1696] Associating HttpWebRequest#38557668 with ServicePoint#62013521
System.Net Information: 0 : [1696] Associating Connection#12798926 with HttpWebRequest#38557668

which meant, there was a problem creating the socket?
whatever - the case continued...

many people were experiencing thi serror:
The underlying connection was closed: A connection that was expected to be kept alive was closed by the server

and offered as solution setting keepalive=false, which i already had in place.
there was also a link here:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;915599
which gave another hint:
playing with ServicePointManager.MaxServicePointIdleTime property

there was also a hint to set
webRequest.ProtocolVersion=HttpVersion.Version10
which i don't to play with right now

the last and seemingly, best solution was given by Darren Neimke
http://markitup.com/Posts/Post.aspx?postId=c98db0ca-f3a8-4f49-bb96-641574274c1a:

WebRequest req = HttpWebRequest.Create("http://foo");
req.ConnectionGroupName = Guid.NewGuid().ToString();

using (Stream stream = req.GetRequestStream()) {
// do stuff here
}

a saw the ConnectionGroupName approach also here:
http://blog.josh420.com/archives/2008/02/underlying-connection-closed-fix-pesky-aspnet-webexception.aspx

still testing....will this help me not see this log below anymore?

System.Net Verbose: 0 : [3828] HttpWebRequest#48833621::Abort(The operation has timed out)
System.Net.Sockets Verbose: 0 : [3828] Socket#3798230::Dispose()
System.Net Error: 0 : [1696] Exception in the HttpWebRequest#38557668:: - The request was aborted: A connection that was expected to be kept alive was closed by the server.
System.Net Verbose: 0 : [3828] Exiting HttpWebRequest#48833621::Abort()

Retrieving the COM class factory for component with CLSID {00024500-0000-0000-C000-000000000046} failed due to the following error: 80070005

2008-06-20T14:12:00.000-07:00

Retrieving the COM class factory for component with CLSID {00024500-0000-0000-C000-000000000046} failed due to the following error: 80070005

seeing exceptions like this make me moan in desperation... spent several hours searching the web...and although i found few good articles on this error here and here, I didn't exactly followed them because in our web app we have impersonating user, so when i opened Component Services->Computers->My Computer->DCOM Config and opened properties on Microsoft Excel Application and would give Launch permissions to this impersonated user.
It didn't work the first time, not the second, so I added this user to the local admins group.
This didn't work much either.
So in the end in some post a line containing 'Event Log' caught my attention and I swiftly opened the event viewer where, under System, found several relevant error messages, one of which

The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID...
gave me the info I needed - turns out I really had to put the ASPNET user account under which asp.net is running.

This solved the problem

I still have doubts as to why the ASPNET account should be held responsible and not the impersonated one, but....whatever!

MS SQL Server Management Studio - Create new table/Add new column

2008-06-19T07:49:00.001-07:00

For some (long) time now I've been irritated by SQL Server Management Studio everytime I add new column to a table that has to be var char with length other than 50, which is the default.
It was like whenever I put value different than 50, Management Studio would revert the value back to 50, and I'd have to change the value again and this time it would work.

Now today I noticed something - this would happen everytime I put a value of varchar(10), but not for other values!
So for varchar(10) it is reverted to varchar(50),
but for varchar(20) the value is preserved when I change the focus....funny, no?

Assembly.GetEntryAssembly()

2008-06-17T08:22:00.000-07:00

I have to use a library designed for use with Winforms.
I have to use it in Asp.net though, so at one point I got to a method using unguarded Assembly.GetEntryAssembly() call that failed.

After a little research and found this in the MSDN:

The GetEntryAssembly method can return null reference (Nothing in Visual Basic) when a managed assembly has been loaded from an unmanaged application

and became evident that the reason for the problem was that the library was call from the unmanaged IIS process.

I searched for a generic solution, but finally only managed to find a hint here:http://www.developersdex.com/asp/message.asp?p=2912&r=5817390

so now I'll add this code to solve the problem

if (null != Assembly.GetEntryAssembly())

{

// proceed with Assembly.GetEntryAssembly()

}

Else

{

// read a setting from the config file and set the entry assembly to this value if not empty

System.Configuration.ConfigurationManager.AppSettings["EntryAssembly"]

}

and thats all for now ...

Linq OrderBy Descending fun

2008-05-19T15:36:00.001-07:00

So for a few days I've been playing with Linq with production code and it looked great and (at least the usual stuff) pretty straightforward.
The underlying c# is scary of course, reminding me of the nightmare from the C++ generics/COM days. But at least now the IDE has evolved and IntelliSense works nicely.

Anyway - I came to the point were some tweaking was needed and some parametrizing - mainly i needed the order by to be runtime desided - 'dynamic' if you wish - so i started reading around blogs and sample code, but all seem to be aiming too high and I didn't have much time to really dive into Linq (which I should do someday anyway)... still quantity changes lead to some understanding of the problem in question , mostly after looking at (not really reading) samples at http://www.ddj.com/database/205604421, http://srtsolutions.com/blogs/billwagner/archive/2006/03/29/ordering-linq-results.aspx
and http://community.bartdesmet.net/blogs/bart/archive/2008/04/27/q-is-iqueryable-the-right-choice-for-me.aspx

I came up with this really small snippet of code that solved easily my problem

public static OrderedEnumerableRowCollection
OrderByEnhanced
(this EnumerableRowCollection source, Func keySelector, bool ASC)
{
if (ASC)
return source.OrderBy(keySelector);
else
return source.OrderByDescending(keySelector);
}

ain't it funny - it is so simple, but looking for a solution to my problem lead me to reading a lot about Linq and helped me learn a lot too, so that time was not waisted, so to say

cha!

OUTPUT clause in INSERT/UPDATE/DELETE statements

2008-04-16T12:07:00.001-07:00

just playing with the new (for sql server 2005 and for me) OUTPUT clause.
I had some hopes for it but turns out i cant write something like

insert targettable (col1,col2)
output Inserted.col1, sourcetable.somecol
select somecol1,somecol2 from sourcetable

that is - i cant output values from the Inserted and also values from the source table :(
a bit disappointing in a way, of course, because i wanted to have backwards mapping between the source and inserted tables, but now I just decided to use one of the columns in the inserted table as this mapping ids placeholder

voila...

EDIT:
according to msdn article on T-SQL Delete
it is possible to OUTPUT exactly what I need...

DELETE Production.ProductProductPhoto
OUTPUT DELETED.ProductID,
p.Name,
p.ProductModelID,
DELETED.ProductPhotoID
INTO @MyTableVar
FROM Production.ProductProductPhoto AS ph
JOIN Production.Product as p
ON ph.ProductID = p.ProductID
WHERE p.ProductModelID BETWEEN 120 and 130;

strange...now I'll have to achieve the same results somehow...

Server: Msg 7347, Level 16, State 1, Line 1 OLE DB provider 'SQLOLEDB' returned an unexpected data length for the fixed-length column

2008-04-11T12:07:00.000-07:00

I got the error described here http://support.microsoft.com/kb/920930
when running a test select * from a view on a linked server

Server: Msg 7347, Level 16, State 1, Line 1 OLE DB provider 'SQLOLEDB' returned an unexpected data length for the fixed-length column '[LinkedServerName].[DBName].[OwnerName].[TableOrViewName].ColumnName'. The expected data length is n, while the returned data length is m.

when searching on google web/groups the compains were for sql server 2000 and below so not much help until i decided run the same query on the linked server itself and it went through without a problem.
turned out the column in question (the one that reportedly returned larger data than expected) is empty.
so i just rewrote the query to return only a subset of data i needed and all worked nicely

so...fun...

/LM/W3SVC/1/ROOT/AppName is unavailable

2008-03-25T07:56:00.001-07:00

So I created this neat c# web service and went to deploy it to our clustered environment.
Before that I ran the web setup on a dev server and it installed flawlessly so what a big surprise it was that when i tried to install it in production I got this lovely message:

(it says: 'the specified path 'lm/w3svc/1/root/yourappname' is unavailable. the internet information server might not be running or the path exists and is redirected to another machine. please check the status of this virtual directory in the internet services manager')

don't you love it when all is fine while you develop but when you try to deploy something maddeningly breaks...

Few hours of google search didn't bring any solutions, so i had to figure it out myself, being the genius IIS admin, that i am (not).
first i thought this might be due to our production servers being in a cluster, (mis?)lead by the '...is redirected to another machine' part of the message, but even taking one or the other of the servers off the cluster (temporarily) didn't help.
after a few similar guess/error tries i simply created a folder in inetpub where my web service would be dwelling, created a virtual folder pointing to this empty folder, run the setup again and.....it didnt break this time(!) - simply copied the contents of the web service and off we went

so i guess Okam's razor works...even for deploying c# web service solutions in IIS

lovely ;-)

.net, soap, web services and enums....

2008-02-19T10:02:00.000-08:00

imagine you have a class like this one

public class BaseClass
{
public BaseClass() {}

protected TheEnum enumcho=TheEnum.First;
public TheEnum Enumcho
{
get { return enumcho; }
}
}

where TheEnum is some enumeration.

now, if you want to use and export this class in a webservice, you would decorate it with [Serializible] and expect everything to work just fine...
so thought i until i discovered that the class generated for the proxy would look like

[System.....]
public partial class BaseClass
{
}

yes, you saw that correctly - its just an empty class, and something like this is enough to drive one crazy...
took me long 2-3 hours to find a solution, and it was hard because as always - the oracle (google) is only giving helpful answers when you're asking the correct questions

.net serialize enum?
c# enum soap?
enum how serialize soap c#?
enum not serialized web service c#?

and a big load of similar ones, to not much help until on one of the searches i came upon this one, where bruce johnson wrote about his problems playing with java/.net web services and especially this one where 'Wim' had problems serializing enum fields. and that probably wouldn't be of much help either had he not mentioned this weird thing: "all fields are serialized, except the enum field...strange thing is, when i rename the member field, it does get serialized" - this was enough to get me started to look for the crazy explanation for the problem (remember?-when all the logic conclusions lead to nothing its time to look for the illogical answers...)

and i finally found it when i added a setter method to the property, like this...

public TheEnum Enumcho
{
get { return enumcho; }
set { int i=0; }
}

to my relief this solved the problem and the property was finally added to the definition of the class in the proxy....breathe...and relax....3 hours waisted...

hope this will be helpful to somebody else too :)

idisposable implementation

2007-07-01T15:27:00.000-07:00

just needed to implement idisposable so i can use a class in a using clause and came upon this link
looks interesting and useful

Parser Error - Could not create type 'XXX'

2007-06-30T07:17:00.000-07:00

i was creating a small test web service.
when i wanted to test it, but adding reference to another project, i got this error in the wizard 'Parser Error - Could not create type 'XXX''
and it suggested that i check the proxy settings blah-blah-blah
a quick search on google got me this, but i quickly found out the actual reason when i went to the iis settings - my test service was in .net 2.0, while the settings for the virtual directory were for .net 1.1
after i changed this all went smoothly....

just 2c...

mysql connector/net and cyrillics

2006-03-09T22:54:00.000-08:00

im writing this application that stores client data in mysql 4.0.18 and i see that any data in cyrillics or unicode is stored as ??????.
wtf??
i search and search the web for solution and found .... nothing !?!?
i remembered i had a similar problem once with software for another client, and at that time a had given up and used MySQL ODBC instead.
this time i decided to continue the search and found this post (http://bugs.mysql.com/bug.php?id=10870) and a little bit somewhere here.
so the solution is to add charset=utf8 to your connection string.
why is it so hard to find the solution to a problem that should be common ? or, hmm, ok - maybe i really should've read the mysql manual properly...

building pwlib

2005-06-14T08:04:00.000-07:00

i have to add 729 codec support to my application. for that i downloaded voxgratia's patched version of openh323 and pwlib and started the fight for building them.
what i kept getting was some unresolved externals, so after a long, long struggle i noticed that the compiler-encoded method string which ptlib looks for for linking in ptlibd.def differs from the string in the ptlibsd.lib which is being linked.
so i decided to replace the strings in ptlibd.def with the correct ones in ptlibsd.lib. everything compiles fine so far, i wonder if it will also work fine ....

file upload with asp.net

2005-05-02T10:53:00.000-07:00

while playing with some basic asp.net stuff i got this

Maximum request length exceeded.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.Web.HttpException: Maximum request length exceeded.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.


Stack Trace: 




[HttpException (0x80004005): Maximum request length exceeded.]
   System.Web.HttpRequest.GetEntireRawContent() +895
   System.Web.HttpRequest.GetMultipartContent() +58
   System.Web.HttpRequest.FillInFormCollection() +256
   System.Web.HttpRequest.get_Form() +50
   System.Web.UI.Page.GetCollectionBasedOnMethod() +70
   System.Web.UI.Page.DeterminePostBackMode() +47
   System.Web.UI.Page.ProcessRequestMain() +42

solution was found here: simply increase HttpRuntime maxRequestLength in machine.config

But I bet you knew this already, didn't you ? :)

knoppix 3.8 and usb modems

2005-05-02T04:41:00.000-07:00

so i decided to get a try at linux and got a knoppix cd. successfully changed the language to english (by default the cd boots in german) and wanted to browse the net. but i couldn't. rebooted to windows and started searching for a solution. after about 3 hours on almost every forum i search everybody said that it is either impossible ot very difficult to get an usb device running under linux. wtf ?
that got me even angrier so i did what i've should've done from the start: went to the source and searched the knoppix site for questions and luckily through reference from another post got to this thread:

http://www.knoppix.net/forum/viewtopic.php?p=82705.

initially that guy's advice didn't work for me, but was great for a start.
i installed the usbnet package and as this wasn't enough i remembered having read somewhere about the tulip package and that it was somehow connected with usb devices on linux. after that i run the network card configuration and .... voila! welcome internet! :) the only remark is that for broadcast 192.168.1.255 should be used, the network card wizard came up proposing another ip...

[Access Driver]Cannot open any more tables

2005-03-05T04:51:00.000-08:00

I'm writing this app which reads heavilly from an access 'database'. all goes well, but after a large dataload, the odbc driver began throwing this message at me! how dare you, little woodpecker! ;)
i paniced at first and even began considering whether i should move to ADO.NET (the app is written in c#). i searched and searched the net (Google) and a note in a
message got me startled - " ...and then close the statement object once the result set query has been accessed..." and this - "..You may also want to close the result set as you say after you are done with it...".
of course i close my datareaders and without this notion, but this constant repeating of "close this, close that" got me an idea - and then suddenly i noticed that OdbcCommand has a Dispose() method. and thats was it! after i close the datareader, now i also dispose of the odbccommand object and everything goes smoothly....

activex - save graphics to file

2005-03-01T09:13:00.000-08:00

i'm writing an mfc activex control and want to save its visuals to a file. so how should i do this? the graphics on the control are created through GDI+, so should i go through Graphics contents to an Image, or may be through CDC?

so far i have this:


CDC* pdc = GetDC();

RECT rect;
GetClientRect(&rect);

Graphics graphics(pdc->GetSafeHdc());
Bitmap bitmap(rect.right - 
rect.left, rect.bottom - 
rect.top, &graphics);

CLSID pngClsid;
GetEncoderClsid(L"image/jpeg", &pngClsid);


bitmap.Save(fineName, &pngClsid, NULL);

but it just outputs a black screen, which is understandable

luckily my search led me to GdipCreateBitmapFromGraphics method and this post:
http://www.pscode.com/vb/scripts/ShowCode.asp?txtCodeId=37541&lngWId=1

where this guy - "Avery" - gives a great hint: it turns out creating the Image this way only gives a blank Image compatible with the Graphics object. to get the visuals all the graphics have to be done on a Graphics object derived from this Image. so i rewrote my drawing procedure to a function to be called with a Graphics object and depending on what i want - i would pass either the Graphics derived from the CDC in OnDraw, or the one needed for saving the image to a file... voila!

openh323.ocx

2005-02-27T06:00:00.000-08:00

i'm implementing a VoIP module for a client and found a very usefull atl control over the great OpenH323 opensource project.
the problems is damian's control doesnt work with a gatekeeper. after much struggle, goolgle searches and code studying, i think i found a solution:
i turns out damian's control doesn't properly initialize its endpoint, thus failing to register the gatekeeper.
so the fix is:
modify MyH323EndPoint's Initialise() method like this -


BOOL MyH323EndPoint::Initialise()
{
  noFastStart = FALSE;
  InitialiseCodecs();
  return StartListener("*");
//original version goes like this: return TRUE;
}

thats it. now the end point properly starts its listeners and you can use a gatekeeper for PC-Phone calls

delegates in java

2005-01-15T10:19:00.000-08:00

so, this c# project that i have to convert to java uses delegates. i hadn't worked with delegates in c# before, so studying the delegates usage in the project was quite interesting. after some research i decide not to use reflection or anything else as a means to implement delegates in java. instead i take a to rewrite the code that uses delegates to simple function calls constructs. delegates here are used to make a templates approach to some tasks - for example visit a hierarchy of objects each of which may have subhierarchy and so on. with delegates used it makes it easy to go down the tree of objects and apply some operation on each of them AND - what is important -the operation would be a delegate so many operations can be executed by calling a single method that traverses the tree. or sort of :)
so my decision - as i cant use template code - is to generally make a tree traversal in a distinct method for each operation.
so instead of

internal override void visitHierarchy(CollectionVisitor visitor)
{
base.visitHierarchy(visitor);

foreach (SomeObject Obj in collection1)
Obj.visitHierarchy(visitor);
foreach (SomeObject Obj in collection2)
Obj.visitHierarchy(visitor);
}

there will be a similar block of code, but implemented for the different operations that need to be executed:

public void visitWithOperation1()
{
super.visitWithOperation1();

foreach (SomeObject Obj in collection1)
Obj.visitWithOperation1();
foreach (SomeObject Obj in collection2)
Obj.visitWithOperation1();
}

and

public void visitWithOperation2()
{
super.visitWithOperation2();

foreach (SomeObject Obj in collection1)
Obj.visitWithOperation2();
foreach (SomeObject Obj in collection2)
Obj.visitWithOperation2();
}

of course there isn't foreach in java... but hope you get the idea....
so this is a little bit tedious approach to rewriting delegates in java, but i hope it works for now

vs.net 2003: Could not copy temporary files to the output directory.

2004-12-22T16:05:00.000-08:00

ok, after fighting this problem for weeks, i gave up trying to have all my project output to the same folder. wrote a copy file post build event and calmed down. the strange thing is this (single output folder for all projects) setup worked very well for months and "suddenly" it started failing - first once at a time, then to an extent where i could not normally work... i read and read posts, blogs, newsgroups, articles, tips and shits with no productive result.
oh, well - after all it is a do-once operation; hope i wont have to fix it again.

mscorwks.dll could not be loaded

2004-12-22T13:59:00.000-08:00

i have sent a new application to one of my customers. suddenly they complain that they recieve a strange message that i have never encountered:
"mscorwks.dll could not be loaded". what the hell ? they have followed my instructions and have installed .Net Framework 1.1 and some additional libraries, so what might be the problem ?? good thing there's google and the .Net 247 community; and this article - http://www.dotnet247.com/247reference/msgs/41/207197.aspx.
so i have mixed a debug and release version ? hmm but why didnt i get that message locally then ? the Mystery of the Sith.....

Edit: so, after i read this thread i began to get suspicious and checked with my clients exactly what version of the .Net Framework they had and.... voila! they had 1.0.xxxx when 1.1 was needed! man, this remote work is getting so confusing sometimes....

set the course

2004-12-17T04:58:00.000-08:00

so i finally made my mind on how to proceed with converting the application from c# to java. a helpfull resource was Infragistics' experience. so for now my goal is to have all the c# classes into their java counterparts, with limited functionality in lots of places in order to only have the things moving... you know - the hardest thing is to get the wheel spinning (or was it the ball rolling? :) , everything comes to its place afterwards

save sent mail

2004-12-10T03:15:00.000-08:00

as a long time user of yahoo mail i was used to "copy message to sent folder" or whatever option. recently i noticed that yahoo mail team have removed this option from the compose email window. a quick search in google sent me to a yahoo mail support page, where i learned that there is new option that would only allow me to save all my sent messages. i used to only save some important messages and now they want me to save all or no sent emails.
isn't this a kind of stupid ?

windows disappeared

2004-12-09T02:28:00.000-08:00

another white hair on my head - im converting a .Net program to Java and needed some java resources. So I downloaded BIE, which i thought might be usefull, and very quickly decided that it is not. however when i tried to uninstall it - Booom ! i noticed that my Windows dir was gone, and something was eating the Program Files folder... crazy...beware of badly written software, kids.. and make backups!... often!

uno posto numero uno

2004-12-09T02:24:00.000-08:00

wow, isn't it a little scary... my first post to this blog. i am gonna use it as an online diary and notes holder for my programming tasks. maybe someone else beside me will find it usefull - wish me luck !