HTTP Server Architecture

From: Dan Lyke 
------------------------------------------------------
So I'm building a better irrigation controller, one that I can
manipulate with my smart phone.

An irrigation controller needs to do timing stuff, keep track of
state, so I need a daemon of some sort. And as soon as I get there, I
start to think "well, rather than having CGIs that talk to the daemon,
why don't I just make it serve HTTP".

And then I think: Rather than copying and pasting one of my older HTTP
servers, why don't I try to make something that's easy to re-purpose.

So I have a C++ object that owns the select loop. Attach servers to
ports, no problem. Server takes a lambda that returns the thing that
responds to that connection, still no problem.

Then we get to HTTP requests. Server creates a request object, sends
it header data. Once request is populated, we can decide what to
instantiate in response via some route (check the method, request
path, Host: header, probably types it accepts, create something that
responds to that request).

But there's potentially this other stuff coming in, from things like
HTTP POST or whatever.

I know there's the "spool everything to memory" (and, in the case of
Form-Multipart, to disk) model, things like Perl's CGI.pm do this.
NiodeJS does "everything is closures and callbacks" which can kinda
work, but...

Does anyone have a particularly favorite architecture for how to
handle the HTTP request/response lifecycle that I can look at? My
target language is C++, but part of the library I'm building in C++ is
the "simplify this so that I can start writing the crap I'd normally
write in Python or Perl and port later in a real language to begin
with" mode.

Thanks!

Dan

Chugalug Digest, Vol 31, Issue 81

From: "Kite, Mike" 
------------------------------------------------------
Thanks Lynn and Meuon.

Putting a pipe between domain and user bears the same bitter fruit :(

It's an OEL6 server, which I think, is pretty close, in this case, to RHEL6.

It does indeed have the return code -6 in /var/log/messages when trying to CIFS mount it.
And I may as well just keep trying smbmount rather than smbget, simply because I'll need to put files there as well as get them.

I'm trying to find the Red Hat article you mention, though this machine does not have a  /etc/request-key.d/ folder.  I do have a Red Hat portal account, I'm trying to recover my pwd to that now.

Thanks
mkite


Small job

From: David White 
------------------------------------------------------
I'd like to consider hiring someone to help me write a program that will be
used on my mail server. I may decide to write the whole thing myself in
PHP, but this isn't the type of work I do best.

I have a MySQL database, mail, with 3 tables: 1 for virtual users, 1 for
virtual aliases (forwards), and 1 for virtual users (actual users on the
system that receive email).

Each table has a primary key value of the domain ID, and the aliases &
users table also have individual primary key IDs.

I need a little web GUI that will allow a user to login to add, delete or
modify existing email addresses (change password).

Currently, all passwords (in virtual

[Hardware] E-ink Monitor

From: Joseph Simoneau 
------------------------------------------------------
On Mon, Jan 26, 2015 at 10:46:15PM -0500, Wil Wade wrote:
> Via OSNews...
> 
> The video shows a fair scrolling. No color, but also powered only by USB.
> If this were $100 instead of the projected $600+, it would be great!
> 
> http://blog.the-ebook-reader.com/2015/01/15/paperlike-13-3-e-ink-monitor-by-dasung-tech-videos/

> 

[Hardware] E-ink Monitor

From: Wil Wade 
------------------------------------------------------
Via OSNews...

The video shows a fair scrolling. No color, but also powered only by USB.
If this were $100 instead of the projected $600+, it would be great!

http://blog.the-ebook-reader.com/2015/01/15/paperlike-13-3-e-ink-monitor-by-dasung-tech-videos/

Mysql Rapid Loading

From: Mike Harrison 
------------------------------------------------------
http://geeklabs.com/widget/mysqlrapidload/Rapid%20Load%20MySQL%20Database

A quick perl hack to split up a large MySQL database “dump” (from mysqldump) into several files, 
and load it using multiple processes. Depending on your data, seems to provide an large speedup. On an 8 hour MySQL load, It loaded in right at 3hours with threads set to 4. 
In this case, everything but two tables loaded in minutes. 

Played with splitting up large inserts into multiple files, but it seems the indexing gets pretty funky. More efficient long term to load a table sequentially (it seems). 






To use PGP, or not to use PGP...

From: David White 
------------------------------------------------------
I'm working on a presentation I'll give next month at a conference geared
towards folks working for smaller Christian / missionary-focused nonprofits
on "Introduction to Security."

Some of the folks who attend my presentation will probably be a 1-man shop
with very little general IT knowledge, and others will probably know oodles
more than I about security and information systems.

Anyway... I installed OpenPGP into Thunderbird a few months ago, although
I've rarely (if ever) used it to sign or encrypt legitimate messages,
partly due to the fact that almost no one that I email uses PGP or have
ever heard of it.

Recently, I've been doing some research into how useful it actually is, and
whether or not it is actually secure.

My findings so far is that the current version of PGP is very secure.

Indeed, according to Wikipedia, there is no known method to breaking PGP
encryption: http://en.wikipedia.org/wiki/Pretty

Program to download transactions from bank websites

From: James Nylen 
------------------------------------------------------
I've written a program to... you guessed it... download transactions and
balances from bank websites.  Most of the bank websites I have seen are
just atrocious in terms of code and design, and I wanted a system to unify
my accounts.

Here's the code and documentation:

https://github.com/nylen/node-banker

I'd like to start releasing more software, so what are some good ways to
publicize your work?

I use this program with GnuCash (free accounting software) and a GnuCash
web interface that I wrote:

https://github.com/nylen/gnucash-django

This combination doesn't do everything I want, but it is still really
useful to me.  If you find any of this useful, great - please use it and
post issues to GitHub.

The transaction downloader is written in Node.js, which I like anyway, but
it's a practical choice too since a lot of these bank websites require
JavaScript to function properly.  So I think this code is also a decent
example of how to scrape pages using a headless browser.

Old code never dies..

From: Mike Harrison 
------------------------------------------------------

Rambling while watching a long slow database load:

I got a strange call last night. Seems a pager company, that bought a 
pager company.. that bought a pager company... that bought a local pager 
company that was the dregs from another local pager company that somewhere 
in the late 1990's I wrote an SMTP mail server in Perl that gatewayed to 
alphanumeric pagers via the TAP protocol.. had a server issue. It's the 
only production C code I ever wrote, and it was munged from someone else's 
example code. Once upon a time it made a lot of money as e-pageme.com
(I used to charge $1 a month for the service, per pager..)

I SSH'd in.. rebooted it.. and its magically working again.

Seems pagers have become the medical worlds secret communications layer..
it's the one way to always contact a doctor, no matter what, because 
pagers are still allowed where cell phones are not.

That project taught me several things, including that most "magic" is best 
performed as a service for a small fee. It also reminds me that some 
things never ever die, even when you want them to, and that I should do a 
better job of making sure what I do is the right thing to do, and done 
well. I'm a big hypocrite on that point, most of what I do for a living 
barely qualifies as duct tape and zip-ties, although time has proven 
most of my kludges to be enduring.

It also reminded me, talking to the technical guys at the latest owners of 
this abortion... that few people understand the lowest levels of 
anything.. the "magic" that is a protocal like TAP
(Telecommunicator/Telelocator Alphanumeric Protocol).
http://en.wikipedia.org/wiki/Telelocator

silly perl one liner to randomize passwords

From: flushy@flushy.net
------------------------------------------------------
I was bored yesterday and a FB status spawned this:

sudo cat /etc/shadow | perl -pe 'sub r{$l=shift;open(R,"base64  
/dev/urandom|")||die$!;sysread(R,$R,$l*4);close(R);$R=~s/[^\w]//g;substr($R,0,$l)}@P=split/:/;if($P[1]=~/\/){$s=&r(8);$p=&r(16);$P[1]=crypt($p,"\\$$s");$

Atlanta linux/unix admin

From: Matt Keys 
------------------------------------------------------
The details are below.

Regards,
Matt

-----
From: Kathy Dean
Sent: Wednesday, February 12, 2014 1:51 PM
To: Matt Keys
Subject: Unix / Linux Administrator - On Call - No Travel Atlanta Growth!

Hi Matt,

I have a client that would like to interview you for a role that they =

need filled. Great benefits and growth opportunity! They are seeking =

people who want to grow into management. They are also looking for a =

culture fit - very team oriented. This is the detailed Job description. =

Preference is someone with exp. with Nagios. This firm is building a SOX =

IT Hub for themselves in Atlanta and the opportunity for growth is =

tremendous.

Qualifications: =95 Previous experience as Linux/Unix administrator preferr=
ed
=95 Experience supporting IMAP/POP/SMTP email a must
=95 Must be able to compile/install/configure server software (apache, =

openldap, php, etc)
=95 Strong knowledge in setup and deployment of database
=95 Experience in managing large database sizes a plus (MySQL, PostgreSQL, =

Oracle)
=95 Strong knowledge in optimization and interrogation of databases
=95 Experience in data integration and database design
=95 Experience with monitoring logs to identify problems
=95 Deeply familiar with networking including installation/configuration =

of routers, switches, firewalls, desktops
=95 Powerful Linux Shell Skills (use grep, sed, awk, find, etc)
=95 Skilled with scripting languages such as PHP, Ruby, Python
=95 Comfort with HTTP service architecture, optimization a big plus
=95 Server scripting tools including perl/bash/sh/tcsh/csh or some =

combination required
=95 Knowledge of DNS, preferably BIND
=95 Experience with Phone Systems such as Asterisk and Hylafax Servers =

preferred

Thanks,
Kathy
Kathy.Dean@TeneoTalent.com
303-443-6520

LaTeX copy and pasting from pdf output

From: "Robert A. Kelly III" 
------------------------------------------------------
 When I generate PDF from a LaTeX source file with pdflatex on Debian, I
run into several issues when copy and pasting from the PDF.

1. Double quotes show up strangely, or not at all. When I copy and
pasted on my system, the quotes are simply missing. When I sent it to
someone else and they copy and pasted and sent edits back in a docx
file, they had \ and " for the double quotes.

2. Indentation is lost, which means paragraphs are not properly delineated.

3. Line breaks are preserved, which is not desirable if you are pasting
into a context where you want the lines neatly rewrapped.

4. Automatic hyphenation is preserved, which is really undesirable if it
gets rewrapped when you copy and paste.

5. Page numbers are copied and pasted with the the text, which is also
frequently undesirable.

If anyone knows how to change any of these things in the PDF output, I
would appreciate it. Ideally, I would like to be able to copy and paste
from the PDF and have all of the characters show up in a usable form,
even if they are ascii equivalents, preserve indentation, but omit line
breaks, automatic hyphenation, and page numbers.

OT: Unix Administrator in Atlanta

From: Matt Keys 
------------------------------------------------------
Forwarded message and job description below that :

---

From: Dana McSpadden 
Sent: Monday, January 13, 2014 11:51 AM
To: Matt Keys
Subject: Infosys McCamish - Job Opportunity

Hi Matt,

My name is Dana McSpadden and I am a Recruiter with Infosys =

BPO/McCamish. We are currently seeking an experienced Unix/Linux/HP-UX =

Administrator to join our Data Center Services organization at our =

Atlanta, GA office. A job description is attached for your review.

Please let me know if you are interested in learning more about this =

opportunity. To express interest or to answer any questions you may =

have, I can be contacted via email at Dana

Paperless office/PDF/Document Management solutions

From: Dan Lyke 
------------------------------------------------------
We run an Ubuntu household. Paper is swamping us. We want to go to
paperless archives. I have almost convinced myself that current SANE
support for the Fujitsu ScanSnap iX500 is sufficient, and that I can
get "scanbd" or "scanbuttond" working with it. Run the output from
that through gocr, get some probably extremely noisy text files, along
with PDFs, I'm headed the right direction. [1]

The question then becomes "what do we do with it?"

I see 3 options:

1. A shared tree that we can search and that gets backed up.
2. git archives checked out on our various laptops
3. Some dedicated paperless office software solution.

#1 is appealing, except for two considerations:

First, what network file system do we use? I'd like something that
works from both inside and outside our network (or at least doesn't
hork up a hairball when we try to boot our machines outside our
network). Should this be:

*  NFS? (Seems like it breaks and requires new configuration with
every upgrade, definitely wouldn't work outside the network)
* SAMBA? (Never gotten this really working well, and putting this on
our server which has outside exposure scares the crap out of me)
* sshfs?

Second, what's the best technology for searching those .txt files that
accompany and were OCRed from the PDFs?

Any suggestions?

Thanks!

Dan

[1] http://www.gaggl.com/2013/08/paperless-office-on-a-budget/

My own private NSA

From: Dan Lyke 
------------------------------------------------------
Short version: I wanna snoop on DNS traffic on my network, and I want
a little auditing to reassure myself about what sort of traffic the
devices on my network are generating.

Background: Alan, Charlene's developmentally disabled brother is
staying with us, and we got him a $79 Android 4.1 tablet+keyboard
combination off of Nomorerack.com for Christmas. If, 20 years ago,
you'd have told me that I could buy a multi-core GHZ+ device with that
resolution 3d accelerated for $80, I would have called it unbelievable
science fiction.

I'm sitting in my cube this afternoon and one of the ops guys walks
over and says "our malware DNS page just got the most interesting
delisting request from your home IP address..."

Indeed. The de-listing request form got a couple of "overseas
pharmacy" and "cheap auto insurance" HTML and BBCode posts from, from
a user agent of

  "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0"

I call home, Charlene disavows running Windows. I log in to the Unifi
controller and don't see any Windows machines on the wireless net.

So I haven't had too much chance to debug yet, but I suspect that this
is coming from Alan's tablet.

What I would like to do is to run a logging DNS proxy on the home
server, and, probably put the DHCP server on that machine as well.

Anyone got experience with the right way to do this? I could easily
whip up a Perl DNS proxy, and probably just install isc-dhcp-server,
but if someone's got some experience with this that'd be awesome.


Thanks!

Dan

Updates usually okay. Never upgrade.

From: Unkmar 
------------------------------------------------------
Can't:
  Install: Python modules, perl modules, C libraries
  Resolve: C libraries.

Upgrades don't work!  Never ever ever ever upgrade.
  At least one thing breaks and it is usually a core issue that breaks many
other things.
Update? sure, why not.  Rare problems do sometimes occur.

I'm going to rebuild.  Fresh install and transfer my data back over.

Lucius L. Hilley III

Software Engineer openings in NYC (revised)

From: Dee Holtsclaw 
------------------------------------------------------
Bloomberg is looking to expand the programming staff in SSEOMS and STEP
here in New York.

The full job posting information is available at:
http://jobs.bloomberg.com/job/New-York-Trading-Systems-CC%2B%2B-Experienced-Software-Developer-Job-NY/21141500/

I started as a contractor in September 2012 and accepted a full-time
position several months ago. Besides C++, we use SQL, a lot of
Javascript for UI and a smattering of Python, Perl and shell scripts.
The benefits are incredible and employees are treated VERY well. The
downside is you have to live in NYC. Please feel free to drop me a line
off-list for more information or just to shoot-the-shit.

=?utf-8?b?KOKVr8Kw4pahwrDvvInila/vuLUg4pS74pSB4pS7KSBh?= =?utf-8?b?bmQgbm93IMKvXF8o44OEKV8vwq8gKHdhcyDilIziiKnilJAo4pejX

From: Dan Lyke 
------------------------------------------------------
So if you're a Perl CGI module user, the flag you're looking for is:


use CGI qw/-utf8/;

(unless you're processing file uploads, but if you "perldoc CPAN" and 
search for "-utf8" you'll find the other magic)

That is all.

Dan

Road Server Project

From: Mike Harrison 
------------------------------------------------------

Nancy and I bought a new RV yesterday, the next project will be turning
a netbook/notebook into an access device and server.

My goal is that it'd provide a consistent Wifi interface and network 
inside of the RV for a tablet/phone and a couple of laptops.
I've been collecting gear including a high range wifi antenna (a 
directional and an omni) - the goal is to merge a combination of WiFi
that I can suck (legally and with permission) and one (or more)
cell uplinks.

I've done some googling, and found some commercial "all in one" boxes for 
this, but me, being me, would love to roll my own on Linux.

My questions for the group:

   Am I insane? (probably)

   Is this a Raspberry Pi-ish project (That might be fun)

   Anyone seen any magic for managing multiple changing upstream 
connections or am I having fun with iptables and routeing commands via 
some perl scripts (which I know how to do..)?  It's been a while since I 
did these types of things.

--Mike--


============================================================
Mike Harrison   bogon@geeklabs.com  cell: 423.605.6943

Being careful with devices with memory chips

From: Mike Harrison 
------------------------------------------------------

Not exactly a direct answer to an off-topic question:

I've been told that devices with memory chips can be destroyed by =
Microwave ovens.. HERF (got a 5watt plus CD Radio?), high voltage like a =
TV flyback transformer or from a spark plug wire, being near or part of =
a plasma torch or arc welder circuit..  In fact, while using my plasma =
torch tonight, I put my cell phone on the hood of the car about 30 feet =
away. Cats and balloons making static electricity can be fun, and =
dangerous for your electronics.=20

You want to be real careful with mobile phones, tablets, SD cards and =
USB drives that you don't do any of those things, it'll erase them or =
fry them without a trace to let you know what happened.  Even the NSA =
would have to grind down the chip and attempt to read the last state =
with an electron microscope or something else even weirder.=20

So be careful out there: iOS, Linux and Winders portable devices are =
still subject to e-brain implosions if handled improperly.=20