linux/ubuntu Archives - developed.be

If you want Laravel to show cached content from Varnish on public pages (so without a cookie), but still want to use a cookie on admin pages, and switch between them, config the following:

Put every admin page on a subdomain: admin.mysite.com

in routes.php add the following:

Route::group(array('domain' => 'admin.mysite.com'), function()
{
//admin routes
}
 
Route::group(array('domain' => 'www.mysite.com'), function()
{
//public routes
}

Set cookieless session for public pages

in app/config/session.php

  • Set ‘driver’ to ‘array’. The option “array” will not write cookies. This is what we want for the public pages.
  • Set ‘cookie’ to a decent name.

Leave everything else default.

Override the session driver for admin pages.

The Laravel Session is initialized at the very beginning of each webserver request. There’s no point in overwriting the session driver in a controller or in a route filter (as strangely suggested on the github) because the session is already loaded and initialized before the route filter kicks in.

To overwrite the session config, you have to edit bootstrap/start.php

In bootstrap/start.php

Right after this line

require $framework.'/Illuminate/Foundation/start.php';

write a code snippet that looks like this:

if(\Request::server('HTTP_HOST') == 'admin.mysite.com'){
    Config::set('session.driver', 'native');
}

By doing this we change the session.driver to “native” (so with a cookie) for the admin pages and not on the public pages.

There is one potential pitfall:

On your admin pages, every asset (css, js, image) must be called from the admin subdomain (except assets from other domains or the cloud).

On your public pages, not a single asset (css, js, image) should be called from the admin subdomain. (so don’t use a “http://admin.mysite.com/images/login.gif” on a www.mysite.com page)

Otherwise, if an assets happens to be a 404 and goes through the webserver, it might conflict or create unwanted cookies.

The above example is a stripped down version of my own implementation. You should care for authentication (I use the Sentry2 package for Laravel). With Sentry and the above setup, you also have to put the login page (or certainly the POST-action) on the admin subdomain. Otherwise the login won’t work (because it will try to write an authentication cookie on the public pages, but can’t because of the “array” session driver, so the user will never be able to login).

There might be other ways to accomplish the same result but this setup definatly works.

These are some wild thoughts about Open Source, what it is, and what is should be.

Some trends:

Open source is the new demo

Companies used to make private software, but now they tend to create more open source. Though the open source product is only maintained by the company and is used as a step-up to the paying version.

Take OpenX for example, a package for online advertising. It comes in two flavors: an open source version (the original) and a priced private version. The open source version is far less superior and less maintained than the private. That idea is inherited from the demo-age: a demo was a free version that missed the features to be useful. Today the demo version is licensed as open source, but just because open source is popular.

The open source package isn’t made to be perfect, no, it’s only made to get people warmed up for the paying version. (in terms of OpenX: the open source version has many security holes, which makes it hard to consider).

Open source is company karma

Companies get popular by releasing their open source libraries next to their private software. I may be cynic, but I feel these packages are only made for company karma. A lot of companies sponsor open source projects only to gain karma from the community and eventually sell their services to them.

Because every company wants to have their own open source library, instead of contributing to a library from somebody else, you create a wide field of all sorts of packages that might be abandoned as soon as the company loses interest. The real, well working open source projects are the ones that are supported and used by a wide range of people over a long period of time. These are not the ones that are created because marketing told us so.

I hear you thinking, if private companies want to contribute to open source, why shouldn’t they?

When MySQL was sold to Sun, they didn’t know that Oracle would buy Sun. Widenius, main-developer of MySQL, tried to avoid that Oracle would takeover MySQL at all costs. Right before Oracle bought Sun, he forked MySQL into MariaDB. As soon as Oracle bought MySQL they started adding closed source modules. So there are 2 software packages that are about the same: MySQL owned by Oracle which is partly open and partly closed, and MariaDB owned by Widenius, which is entirely open.

The danger of open source bought or created with private money, is that it might be transformed into closed source software or be taken away from the community. The open source version could be stopped, put on low priority, or be degraded to “demo”.

These moves also cause confusion amongst users. Should they use Open Office or Libre Office? And do they care/know what the difference is? And what about organizations that use an open source package which suddenly transforms into closed source?

The idea behind open source (or community initiatives like Wikipedia or non-technology ones like Transition Network) is: you take from the community, you give to the community. Not necessary in terms of money, but in terms of your skills and your time – whatever your skill may be. Most initiatives need money, so money will be welcomed, but your input is of most importance for the success rate of the project. Wikipedia needs money to run its server and pay its few employees, but even with that money they wouldn’t have made it without the help of all the voluntary writers and readers.

Forks create chaos

The open source community splits into branches. Splitting into branches is a human thing that has been around since the beginning of politics and religion. Splitting up creates quantity but not quality. Just take a look at the discussions about Unity, the new desktop layout of Ubuntu. A part of the community solved it by suggesting another Ubuntu that didn’t implement Unity: Linux Mint. And while Linux Mint is great (I use it daily), why couldn’t we simply agree to stick to Ubuntu and implement the option to disable Unity. It’s open source so it’s possible.

This is where Open Source should make the difference with Microsoft. Microsoft did an equal move by removing the start-button and implementing a dysfunctional desktop (Metro) without any way to “change back to normal” (while Windows users crave for a solution to make their pc’s go faster and don’t care about a new desktop).

Instead of creating one successful well supported product, we create forks, versions that are just slightly different than the original.

All these branches, “doing this different because we believe it’s better” make it impossible to maintain oversight. This is the comparable to Microsoft trying to push their “standards” just for the sake of having an own (in their case: patented) standard.

There are dozens of ways (libraries) to upload a file on a website. If I really want to have the best solution, I have to go to all these projects, demo it or install it. It would be better to have one or two projects that are flexible and well supported by all browsers. Developers just have to learn working with 2 packages and can start working for any employer or follow up any project. It could be taught at school, it could be far more popular and better than any of the dozens of libraries today.

jQuery kind of goes into that direction by creating 1 flexible good javascript library that is wildly supported. But the jQuery-libraries by 3rd party developers make it a mess. There’s no oversight in all these modules, the quality is very different amongst projects, they could conflict with each-other or not be compatible with a new/old version of jQuery.

This is the real pain: “wild” libraries as opposed to “well supported” libraries. This is what gives open initiatives a bad name: the lack of equal quality. Because everybody can create open source, there’s no control, hence no quality assurance.

I am well aware of that contradiction. It’s a debate: do you allow anyone to contribute (democratic) and risk quality instabilities, or do you select the contributors that probably will assure quality but make it less open?

What to do with “bad” contributors/modules?

At my job, an alternative online newspaper, we have a comparable problem. Many of our writers are volunteers, some of them can write good articles, some of them don’t. But what do we do with bad writers? There are 2 schools of thought:

1. We allow bad writers to continue an open democratic website where everyone can report what they want, with the risk that bad articles can harm our quality level (and reputation). Bad writers take a lot of time and effort (it’s more work to rewrite a bad article than to write a good article yourself).

2. We only keep the good writers. That would transform our website into a closed medium and conflict with our basic ideas. By maintaining a high standard we could scare away potential new volunteers who think they’re not good enough but might be.

Keep in mind that some volunteers are bad writers but have interesting things to say. Though, there aren’t enough resources to train every volunteer who fits that category.

We’ve discussed this for hours and it’s hard to figure out a middle way. Currently we have to idea to “star” contributions which we think are good, a quality label. We only want to make that clear with layout changes, because we don’t want to add a textual “warning-this-article-sucks-disclaimer”. That kind of disapproval would make the volunteer displeased, if not angry.

I think that idea would work for Open Source as well, and some of them have started such an idea. Drupal contributors, for example, start with a sandbox project that has to be reviewed by an admin. If your sandbox is alright, it will be transformed into a module. Too bad, too many modules have features that are just slightly different than another. This confuses people: “what module should I use? Google Analytics Integrator? SEO Expert? or just the module named Google Analytics?

The bigger plan is of most importance

Just “starring” doesn’t work if you allow every module by the simple rule that the contributor must be a good coder. There needs to be a bigger plan:

  • What modules do you want?
  • Are the current modules good enough?
  • Which modules should be replaced by better ones?
  • Who wants to manage that?
  • Do we allow everyone to contribute? Or how will we select?
  • Is the project “owned” by a private investor? And do we allow that?
  • How do we collect money in case we need it?
  • How do we get people to contribute?
  • How do we handle requests for certain modules that might not fit our software?
  • Do we risk losing users by not implementing certain features or do we implement everything just for the sake of attracting as many users as possible?
  • Who will decide what to implement? How is that process defined?
  • How do we handle bad content/contributors?
  • Is their a “leader”, someone who pulls the strings? A decision maker? And if not, how do we organize?

I know this comes scarily close to management, but these are questions any serious open project will have to answer some day. It would be a pity if open source projects fail by not thinking these through. These type of questions should be answered for every community project, and not just tech ones.

The reason I think why these questions are left unanswered, is because it’s not a pleasant task and it doesn’t add production value right away. If I spent one week thinking about the questions, I loose one week of coding. And, maybe my time is limited to one week. In case of open source, most contributors are developers. And developers want to develop. They don’t want to waste time on the above questions, no, they want to code, rather now than tomorrow. Many developers, like me, don’t like to “manage”. They get behind their computer, start coding, and hope someone will spontaneously say “hey, can I contribute?”. That someone would be a great coder with the exact state of mind as ourselves, and not some sucker who just created his first html-page.

If I look deeper into myself, the thought that someone would “take over” my project, scares me. That’s perhaps another reason why some questions don’t get answered. If other people involve, I could loose the project, my name in bold on the about-page.

Of all the payed web-projects I left, every now and then, I check back on that site to see how things went on. What did they implement? What did they cut? How did they handle that complex js-problem?

It happens that nothing changed at all: the bug that was reported 5 years ago is still in it, the “temporary” solution has become older than my cat, and the space looks frighteningly… dead. Is this what I created? Did someone forget to turn that server off? Is it all forgotten?

Or, the other side, the project is gone, replaced by something flashy else, dumped on a backup harddisk in a basement.

Luckily, most of the times, the project appears to be in good shape, nice features have been added, developers clearly knew what they were doing. It has been handled respectful. This is what well managed open source projects should become. This is why the questions are important.

I better start thinking about the questions right away but first I want to code that feature that will make the project look awesome.

It felt like a Monday morning. After my alarm clock didn’t get off (+ 1 hour), I noticed there was only a train 1 hour later, so instead of arriving at half past 9, I arrived at 11 and missed the first speaker.

Lightswith + Drupal

Anyhow. I picked up the last quarter of using Drupal together with Lightswitch (= Visual Studio). Apparently not a lot of devs had interest in the subject because there were only 20 people in the room. And those who didn’t attend were right, because all the speaker could tell us was that marrying Drupal and Lightswith could only result in a divorce.

Lightswitch can create a HTML5-admin environment based on a data layer. That data layer could be your Drupal database. Nothing works out the box for Drupal because MS of course wants to integrate their software (SharePoint, Office) and not someone else’s software.

Another downsides of all these MS-Drag-And-Drop-Automatic-Data-Layer-Builder-stuff, is that when you change your database, something on the other side might break and you could end up writing the data layer yourself (as a attendee commented). Plus, the actual html output looks weak and is unusable in a serious professional environment. Don’t try this at work, pro’s!

Drupal 8 discussion panel

Three Belgian core-devs (swentel, Wim Leers, aspilicious) had a one hour Q&A hour about Drupal 8. They all had a lot to tell so the number of actual public questions was limited.

You had to know some Drupal 8 in forehand, because new projects (say WSCCI, PSR or TWIG) were discussed without being explained.

The main message was that Drupal 8 is ready to port your modules to. But, there’s still a lot of work to be done. There are still upcoming API-changes, you can’t translate a node’s title yet and there are various other big and small release blockers. But: Views should be finished. Ah!

And why Drupal 8 should be better that its processors:

  • PSR proof. PSR is a PHP coding standard. (aimed for PSR 4 however, it’s uncertain if it the project will get there)
  • Display Suite is now a core module (however, is this really such a big plus?)
  • Getting rid of the hooks in favor for a more object oriented way (however, hooks still exist)

Continue reading “DrupalCamp Leuven 2013, a brief Saturday review.”…

  • Most fonts are located in /usr/share/fonts
  • But there isn’t just one folder. You can find the location of all the font folders in /etc/fonts/fonts.conf
  • You can save custom fonts in the folder ~/.fonts . It’s possible that the folder doesn’t exist, so you have to create it.
  • The filenames of some fonts are different from the actual fontname. Search in the directory for parts of the fontname. Eg: the “Monospace” font in Ubuntu is actually an alias for DejaVuSansMono and is called ttf-dejavumono.ttf
  • sans” or “sans serif” like Arial, is a font with no accents attached to the characters. (sans is French for “without”). Use for screens and websites.
  • serif“, like Times New Roman, is with accents attached to the characters. Use this for printing text or books.
  • mono” or “monospaced” like Courier New, is a font which characters are equally wide. Use this for coding and html.
  • To clear the font cache, like when you’ve downloaded a new font, use this command:
    sudo fc-cache -vf

 

This tutorial explains how to secure your Dropbox files with Truecrypt in Ubuntu (or Linux Mint). It assumes you know Truecrypt already and have a basic understanding of the Unix folderstructure.

Why securing your files in Dropbox?

I use Truecrypt for keeping my personal files. Basically all my important files are in a 50GB volume. My Dropbox folder was located inside the Truecrypt volume.

Like this:

/media/truecrypt1/Dropbox/all_my_files/

I wasn’t satisfied with the system. Who knows what happens with your data when you submit it to Dropbox. A hacker could get access to my account, or a Dropbox-employee or the government (not all stories are conspiracies).

There wasn’t really a point of securing my data with Truecrypt, when everything inside the Truecrypt-volume was copied to “the cloud”.

What even bugged me more were the credentials on my filesystem. Some folders need different credentials (www-root, root-owned files, mysql-files). When Dropbox faces a file it can’t access, it keeps on indexing and consuming cpu.

How does it work?

I came up with the following script:

Continue reading “Secure Dropbox by using Truecrypt volumes”…

To be honest, I’m a bit fed up with Drupal lately. We’re stuck with an ever-growing fat Drupal6 site, and the monster doesn’t get easier to maintain. The question isn’t if we will upgrade, but to what we will upgrade. The logical upgrade to Drupal7 will be enormous because our website has over 150.000 lines of custom php-code (not mentioning css, js, themes). So moving to Drupal 7 or creating something entirely new based on a framework… I don’t dare to say how much difference in time it will make, but I do know that the new thing must be a giant step forward to what we have now.

Why moving from Drupal to a Framework?

As much as I love Drupal, the thing is… a lot of modules do kind of what I want, but they don’t do exactly what I want. Customizing a contributed module sometimes takes me as long as I would write it myself. Especially when there’s little info in the README-file or when there are hardly any comments in the code. To give you an exampe: it’s faster to lookup the url for the admin-page in the menu-hook, than to find it in the documentation or in the navigation.

Plus, what irritates me the most. When you have +75 modules, +100.000 nodes and +10 currently logged in users, the thing gets slow.. slooowww! Even with advanced caching tools such as Memcached,  view-cache or Varnish, the thing still goes slow, even on a dedicated server. No wonder: every hook is checked for each and every request.

So I’m looking for something faster and more OO-like. I have a C#-background and the way classes are mimicked in php (including inheritance, namespaces) is simply terrible. I’d dare to say Drupal goodbye, as long as Drupal 8 isn’t released an is more decent.

My comparison of PHP Frameworks

Pfhew, long introduction, but here’s my research on PHP-Frameworks. Briefly, a framework provides a set of functions and classes to help developers write code faster and more structured. Most of them implement certain design patters, of which MVC is by far the most popular. The aim of MVC is to seperate the database-talking-code (Model) from application-specific code (Controller) from html (View). It also features a URL-mapping system to set “clean url”-rules and to separate the code-files from the actual urls (eg: /bootstrap.php shouldn’t be accessible through a browser).

Some devs use the bootstrap of a CMS as a framework-starter (like my RSS-boostrap), but it should actually be the other way around: a CMS should be build on a framework. Drupal8 will (maybe) rely on parts of Symfony2, but that won’t be out till 2014.

Continue reading “PHP Frameworks, which to choose in 2013? A comparison.”…

Like previous years DeWereldMorgen.be organizes an Ubuntu Install Party. Be present Sunday May 25th in De Vooruit. All details.

I upgraded my Ubuntu 10.04 to Linux Mint 14. After installing LAMP I got a Drupal WSOD on a previously well working site. As it turned out most errors came from deprecated php-functions and deprecated call by references to functions, introduced with the release of PHP 5.4.

At first I was a somewhat encouraged to solve those deprecated functions, but I gave up pretty soon. Drupal 6 isn’t designed for 5.4. Tweaking Drupal feels the same as upgrading to Drupal 7. Therefor, I keep it to PHP 5.3.

To downgrade 5.4 to 5.3 I recommend this script on the Ubuntu forums.

Instead of logging to the database, it could make more sense to log to the operating system.

Why?

Pro

  • The OS has already a great way of maintaining logs, so there isn’t really any need to have a separated database logging.
  • The options to maintain or report logs with the OS are much larger than the default Drupal database logging system.
  • Database logging eats away performance from your server, certainly when Drupal writes tons of logfiles each day.

Con

  • Any shared hosted website can’t use this option, because they have no access to the OS’ logsystem.
  • If your website is maintained by a so called “webmaster” or a “not so technically equipped person”, it may be easier to log to the database, because nontech persons would generally prefer to access the logs through the Drupal admin interface instead of using some dodgy Unix command. Of course, you could ask yourself if a nontech person is anything with the Drupal logs in the first place.

Continue reading “Write Drupal logs to rsyslog instead of to dblog”…

Our main RSS-feed at DeWereldMorgen.be is the most requested page next to our homepage.

It seems logic, think of how many rss-readers hourly check the feed. And, think of how many cpu and RAM that consumes, certainly with a fat system like Drupal.

An RSS-feed is easy to make in PHP. All you need is one custom query and a decent library like the Universal Feed Generator to generate the XML.

Create stripped version of Drupal setup

I used the minimal code that is needed to work in the Drupal framework. So I made a blank php-file in my www-root with this code:

require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

This code has the security and functions of Drupal, but without the menu’s, theme’s, and a lot of module hooks. You don’t need a menu or a theme to create an RSS-feed, do you?

So I created my RSS-feed with just this in my php-file:

  • the drupal_bootstrap function
  • the Universal Feed Generator library included
  • one query with db_query()

With cache

Let’s see performance wise. This is the report of a feed generated based on a user’s blog and articles. Memcache was not cleared before execution. Stats are generated with XHProf.

Normal setupMy stripped setup
Number of function calls115.20018.400
Consumed RAM126 MB62 MB
Total execution time1.169 MS462 MS
Number of database queries3315
Query execution time137 MS80 MS

Performance wise, my solution is twice as fast.

Without cache

These stats are generated when caches were cleared.

Normal setupMy stripped setup
Number of function calls3,284,34383,330
Consumed RAM172 MB76 MB
Total execution time9,131 MS1,000 MS
Number of database queries926131
Query execution time769 MS183 MS

9x as fast. 7x less queries.

Of course, this is just for the first run, but still.

More soon. Please discuss this idea.

Next Page »