We had a situation at work where we had a product search routine which we wanted to optimize a bit. So spending some time looking at execution plans, and we started optimizing the various indexes for what we could – changing scans to seeks and key look ups for what we could and so on.

The jist of the routine was a number of specific searches union’ed into one result. I’m not going to post the SQL because it doesn’t matter much for the given situation as it was more the experience I wanted to convey.

So after optimizing all the indexes we could we were still unhappy with the performance, but didn’t really have the timeframe available to change the architecture of the query or database, we started trying more untraditional aspects. First, because the results of the numbers of unions were varying, but all ending up with the same output (of course), we tried to only select the ID’s out and wrap them in a Common Table Expression, and use the ID’s from that to extract the actual product result.

That however did not help much, so we tried a number of other techniques, to identify the bottlenecks.

The strange thing was what actually worked. Instead of using a common table expression to wrap the multiple unions, we made a table variable and selected the ID’s into this variable, and used that to pull products and product information. This worked much better for some reason and I find the result rather counter-intuitive based on what I have tested and read and know about databases so far…… I guess I’ll have to try and reproduce the situation in an upcoming blog piece to see if I can identify why using a table variable improved performance so significantly as it did in this situation.

The lesson I learned from this was to try all sorts of avenues, instead of being bound by what one thinks is the correct way. That is – do not exclude possible ideas without giving them a trial run first, but I guess that is a very important lesson to actually learn, and I'll do my best to apply it.
That was a strange day

Just wanted to make some advertisement for the company I work for: "hedal:kruse:brohus".
We have just released the financial result (sorry – the press release is in Danish)
for the last year and showed a healthy growth; meaning a growth in business,
number of employees as well as profit.

Granted we are still a "small" firm in the grand scheme of national and international scope, but we leading on our market and we are still going strong with our e-solutions, whether they are electronic catalogue production (a case of a client states they saved 75% on their printed catalogue production by using our solution), e-commerce or likewise.
Plus we are starting to see increased international perspectives and interest in our product(s), so the future looks to be interesting from the perspective of a developer, at least.

I’ve been
reading a large number of technical books over the time.
And one thing which often strikes me when I read these books are how often
examples and solutions are “outside the box” and I always get sort of envious
of how these authors can do that.

Is it an
experience thing( meaning a learned thing) or an innate ability?

If it is
the first then there is hope for us mere mortals because experience and learning can all come with time. When I read books I try my utmost to
look at the reasoning behind what the authors write, and thus I attempt to try
and think “out of the box” because I would like to learn how to do so more
often. And when I code I try to apply that reasoning to my own problems and see if I can do X or Y instead of Z. It is not something which comes natural to me – that is clear – and I
started thinking about why that could be. Of course some of it is a "cost-effective" issue because sometimes you only have so much time to solve a problem and then the tried way can be faster, but it is not especially fun or perhaps even a good way.
It then struck me how much of my educational
processes have been “inside the box” and almost punishing outside sort of
thinking, even all the way back to public school, which somewhat locks you in
that kind of thinking.

Of course
some subjects in an education are purely “inside the box” – how to spell (which
I never learned well) and all that, however many subjects and problems are not
automatically inside the box.

Math for
one. When learning mathematics to begin with back in school, it was “formula X”
for solving this problem.  “You can only
solve a second degree equation using this method”. And well, yeah – most likely
that is needed, but that sort of thought process sticks with you. As I moved to
more advanced math it was similar, although now we also should mathematical prove
that you could (should) use the given formula to solve the problem, and I found few
things more dull then that, taking a formular and "prooving it" – well save analyzing poems and stories.

Rarely did
mathematics education ever try to encourage creativity, new thoughts and all
that. It was all focused on that exam, and getting a grade because you could
remember stuff. It was the same when you should analyze poems for various language
classes and other such texts. I hated it because it was either using the
textbook interpretation or you did it wrong, and come now – what should be open
to creativity more than interpreting pieces of “art”.

This
mentality followed me on the way to university, for the years I studied
economics there. Marketing was “standard” replies from textbooks. Organization
and business/international/micro/macro economies, optimization problems and all
such theory were similar standardish. You had a textbook which explained what to do, how to
do it and why to do it. So you should follow them and get your degree. Sure you
could go “outside” the books and draw reasoning, but the sure way, and worse the encouraged way, was to follow
the books. "It says X in book Y, so that is why…." type response. 

Whenever we
tried to think outside the box we ended up in problems, okay – perhaps our suggestion
to do economical crime, empty a corporation value and move out of the country
was less then optimal, but it was a symptom of the situation and we tried to challenge the mindset. Whenever we –
myself and a couple of fellow students – tried to challenge the “established”
theories, we got smacked upside down. We took conventional theory and applied
new mindset, and sometimes failed miserable – but at least we tried to think
and not just do as we should. But in an education where it is about getting a grade to move
on, there was little room for such experimenting unfortunately.

When I then
started my programming education, it at least opened the possibility for
thinking outside the box, but I could see on my fellow students that it was
often lost due to the indoctrination through the educational system.
When we analyzed problems and drafted design papers, it was for the majority of
time “follow this method” type of work. Then at least, when coming to larger
papers, we found the freedom to think outside the box, however we still
followed the methods. What we did instead was fragment a method and take the
few tools from a method we felt we could use and combined them, with sometimes
little success and other times to great success.  However it wasn’t the “usual” thing to do, and
it was clear that this challenged our advisers and teachers just as much, and
it wasn’t always well liked by them.
The same thing happened with the programming classes which is where I felt it
was the most waste.  You had a problem,
you read about a pattern which could solve said problem, and you explained how
you followed the pattern. Oh gee, now that’s innovative thinking for you.  

So now with
a couple of years of actual programming and problem solving work experience under
my belt, I hope that I can get this “outside the box” mentality down, because
it would make many problems easier to solve, faster, better, more efficient.
But it surely wasn’t something I felt I’ve been trained to do through the
educational process, quite the contrary. But I do feel now it is needed to
actually solve a number of problems, unless one wants to turn to others who do
the thinking for you, and I do not want that. I want to be innovative, I want
to dare be different and I want to solve problems new ways, and not just the old
and tried way.  So, I hope it is
something I can pick up (soon).

Of course,
it might all be an innate ability and then I’m out of luck and will have to
rely on smarter people then me to do the thinking for me…….

I stumbled upon one of my old(er) personality tests, which I took at a XML course back some years ago when I was unemployed.  And I got to thinking about how easy it is to actually just answer the tests on how you want the result to come out.

“But you are supposed to be truthful” I hear as a response… well, yeah, but there is a thing called context.

There is a context of how you want to present yourself, and there is the context – or actually often lack of – in the questions of said test.

The context you function in, decides the outcome you want out of the test. So despite I’m a major introvert when it comes to my personal life, then when I am in specific settings, I want to present myself as an extrovert.  This means that if I am taking one of these personality tests – I’ll try to deduct what kind of persona I want to “show” and I’ll answer given that context.  This means that if I’m being tested in a situation where I feel being an extrovert is more advantageous then otherwise, I’ll score high in that aspect… that is the context.
It is in my view easy to read these tests, and if you just put some care into it, you can form the result pretty much as you want, and nobody will be the wiser. Underhanded, perhaps, but I think most everybody does it to some degree, whether they'll admit it or not. Some people change their behaviour given who's around them for example, and that is the same.

Another issue with many of these tests is that they lack context in the questions.  If one then look at many of the questions some of these tests present, then it is quite clear that the questions also revolve around a hypothetical situation, however it is impossible – in my opinion – to answer truthful on a hypothetical situation without knowing the context: “Do you work well with others”, well duh … that depends fully on who the others are. If it is slackers who do not want to put in any effort but just reap the rewards, then no. But that is difficult to answer.
And such things are in my view fundamental flaws with all these personality type tests, and that is why I have difficulty understanding many institutions focus on such tests.  Many of the statements I’ve read over the years from companies utilizing such tests in an application situation, excuse a focus on them with the argument that it is the only way a company can get to “know” the person before hiring him – and my response usually is that, well they do not get to know the applicant. They only get to know the persona the applicant wants to present.  And even if the person answers “truthful” without attempting to fit into the context of taking the test – the result is rather useless because the context of the questions themselves are mostly lacking (or understated) and thus can’t count for anything in real life.

My guess is that it is just easier to group and box people up if they think the results are quantifiable that easy. But ….. outside fun party tricks, I do not see anything useful in them.

Just to mention, the test I fell over was a Jung type personal test, and I scored INTP.