Tombola is a webapp – a web page that behaves like an application – designed to help teachers, lecturers and trainers to ask questions of everyone in their lessons and seminars.
Categories
Month: November 2017
Categories
Virtualmin admin scripts
The college’s neongrit linux server uses Virtualmin/Usermin to handle the admin of users, and providing those users with a web interface to configure things like how apache works for their virtual host, creating new databases, etc.
It’s a very useful framework, and has most of the essentials we needed, but I wrote a few scripts in Perl to assist me with some frequent tasks, the way that fit me- which is perhaps more true of sysadmin scripts than any other sort of software, that they are built to scratch an itch. Here’s one to assist batch adding of users and creating them a virtual web host.
It’s a very useful framework, and has most of the essentials we needed, but I wrote a few scripts in Perl to assist me with some frequent tasks, the way that fit me- which is perhaps more true of sysadmin scripts than any other sort of software, that they are built to scratch an itch. Here’s one to assist batch adding of users and creating them a virtual web host.
Categories
Grokking the subnet mask
I’ve noticed that many networking students have few problems answering questions about the simpler, classful networks and their subnet masks. But then many struggle when problems involve CIDR networks.
I think it’s because beginners have grasped the basics using some rules of thumb, but not the underlying principles.
When we take the class C netmask, 255.255.255.0, the shortcut thought is that the magic number 255 means “part of network”, and magic 0 means “part of host address”. But if that’s all you know, then as soon as you see a netmask like 255.255.192.0, you’re stumped. What does the 192 mean?
I think it’s because beginners have grasped the basics using some rules of thumb, but not the underlying principles.
When we take the class C netmask, 255.255.255.0, the shortcut thought is that the magic number 255 means “part of network”, and magic 0 means “part of host address”. But if that’s all you know, then as soon as you see a netmask like 255.255.192.0, you’re stumped. What does the 192 mean?
Categories
JavaScript userscripts
Problem
I was playing a browser-based game that required me to perform an action at 04:00. And I wanted to be asleep at 4am.
One thing I want all budding web developers to remember is this- your application front end is running on someone else’s computer. In what is effectively a debug environment. Developers need to ensure there is no trust between the client and the server (the other way around is fine, however).
Because the code is not compiled, it is even easier to dig into the code- the HTML and JavaScript -and make changes.
But I didn’t need to go that far.
Using Firefox, and hitting Ctrl-Shift-K, brings up the Web Console. From here, we can look at messages, errors and warnings, but we can also issue commands in JavaScript.
Solution
[code lang=”js”]setTimeout(function() {document.getElementsByName("act_use")[0].click()}, 120*60*1000);[/code]
Let’s break it down. First, I need to trigger something in the future. That calls for setTimeout() (setInterval() if we wanted to keep triggering again and again).
setTimeout() takes two arguments, which we might call what and when. what needs to be a function- not the result of a function, but a reference to a function, and if you won’t be using it elsewhere, you might as well inline it, as above. The when is the delay time in milliseconds.
The function body will contain all the instructions we want to carry out. I wanted to simulate a button press on a button element. So the next step is to get a reference to that element. Looking at it, I could see it hadn’t got an id attribute, but it did have a unique value for its name attribute. So that indicates use of document.getElementsByName(). Note that it is “Elements” and not “Element”, because name isn’t necessarily unique on a page. So it returns an array, of which I wanted just the zeroth element- so my call needs the index [0] suffixed to it:
[code lang=”js”]document.getElementsByName("act_use")[0][/code]
This expression should now be a reference to the button element. And what did I want to do with it? Click it. Luckily, there is a function available to us, click(). We can just tack that onto the end, and that’s all our function needs to do.
There’s not much to say about the delay, but note how I have used an expression, not just a literal. That’s to allow me to edit it more easily in future. The 120 is the number of minutes, and I can change it directly, as opposed to working out how many milliseconds it would be.
So, leaving my computer awakw, and the browser open, I issued the command in the Web Console, and came back down in the morning to find it had worked like a charm. Itch, scratched.
If you want to explore this further, might I recommend you take a look at GreaseMonkey or TamperMonkey, for permanent scripting you can apply to pages you visit?
Categories
Homebrew Intervalometer
I wanted an intervalometer for my Canon DSLR, which at the time, rather miserly did not contain. An intervalometer is a device to take exposures at regular periods. It’s great for time-lapse work. Their own add-on was quite pricey. Even knock-offs were more than I thought was worth it. And besides, I wanted a project where I could make my own printed circuit board, or PCB.
So I thought I’d build one myself. The fanciest way would have been something to talk USB, which could have made all sorts of changes to the camera settings per-shot, but I went with the ultra-simple “bulb” interface. All you need to do is close the circuit between two contacts, present on an unscrupulously proprietary interface on the camera. A simple switch would work. So would a relay, if you want to automate it. But a relay uses quite a bit of current to operate, and one useful property any intervalometer can have is to run for a long time on a small battery.
So I elected to use an opto-isolator. These devices are totally electrically unconnected to the power they are switching, as they work by the input current switching on an LED inside an IC-style package, which shines onto a light-sensitive transistor, which switches on, allowing the current to flow through it. There is nothing other than light going between the two sides of the opto-isolator, and only in one direction at that!
I experimented to find what the shortest was that I could close the circuit and have the camera reliably take a photo, and wrote a short program to activate the pin that would turn on the opto-isolator for that time, and then sleep until time to fire again. I added an LED to visualise what was going on.
I designed the PCB in, erm, PCB.
This was printed out with a mono laser printer, onto the very glossy, flimsy inserts you get in newspapers, advertising, usually, cash for gold, or takeaways that sell EVERY FOOD EVER. Place this onto some clean, copper PCB board, and apply a hot iron. The kind you do shirts with, not a soldering iron. This melts the plastic toner binder, and it should preferentially stick to the copper. You then wash and very gently clean off the paper, and you’re left with the black toner where you want the copper to remain.
This goes into the etchant, which was hot ferric chloride in solution. Once this is complete, you can, again, carefully, clean off the toner, to re-expose the copper.
Next, drill the holes for the component pins. Highly recommend not doing this by hand, PCB drills are super brittle and will easily snap off. Then, solder the components in place. As a temporary housing, I think I used a box that some cufflinks came in.
The microcontroller was programmed with the following code:
[code lang=”c”]/*
* intervalometer.c
*
* Created: 22/07/2011 22:12:03
* Author: Phil Bambridge <phil@swedeheart.net>
*/
#include <avr/io.h>
#include <avr/interrupt.h>
#define FALSE 0
#define TRUE 1
unsigned volatile int systime = 0;
ISR(TIMER0_COMPA_vect) {
// Given our setup, this being called means a millisecond has elapsed
systime++;
}
void delay (int mssecs) {
int timenow = systime;
while(systime – timenow < mssecs) {
MCUCR |= 1 << 5; // Disable sleep guard
asm("sleep"); // We’ll wake up when the timer counts a millisecond and throws an interrupt
MCUCR &= ~(0|1<<5); // Enable sleep guard again
}
}
int main(void)
{
SREG |= 1 << 7; // Enable global interrupts flag
ACSR |= 1 << 7; // Turns off the unneeded analog comparator
PRR |= 1 << 0 | 1 << 1 | 1 << 3; // Turn off the ADC, the Serial and Timer 1 respectively
// Set up timer for a 1KHz interrupt- for 8MHz operation, that’s a 1/64 prescaler, and a count of 124.
// For 1 MHz (default) operation we’d go with a 1/8 precaler.
OCR0A = 124; // Count to 124
TCCR0A |= 1 << 1; // Clear timer when we hit that count set above
TIMSK |= 1 << 4; // Enable the interrupt
TCCR0B = 0x2; // 1/8 prescaler, also enables the timer-counter.
// Set all three output pins low, then set to output.
PORTB = 0;
DDRB = 0 | 1 << DDB0 | 1 << DDB2 | 1 << DDB3;
while(1) // Run forever
{
// Put shutter and indicator high
PORTB |= (1 << PORTB0 | 1 << PORTB3);
delay(75);
// Put shutter and indicator low
PORTB &= ~(0 | 1 << PORTB0 | 1 << PORTB3);
delay(2000);
}
}[/code]
By way of comparison, if I’d done this with an Arduino, the code would look something like this as a close approximation- it’s basically the blink sketch:
[code lang=”c”]#define OPTO 12
// the setup function runs once when you press reset or power the board
void setup() {
// initialize digital pin 12 as an output.
pinMode(OPTO, OUTPUT);
}
// the loop function runs over and over again forever
void loop() {
digitalWrite(OPTO, HIGH); // turn on the opto-isolator (open camera shutter)
delay(75); // wait for 75ms
digitalWrite(OPTO, LOW); // turn off the opto-isolator (close camera shutter)
delay(2000); // wait for a 2 seconds
}[/code]
As you can see, it’s a lot fewer lines of code. The Arduino preprocessor builds the real code after you hit the compile button (if/when you know C, you realise that what you see in the Arduino IDE can’t be the whole story), we can find the actual code in a temp file. In this case it was 8036 lines, 90% of which were comments. Not a bad read for a lazy Sunday afternoon. Point is, it’s doing lots of things behind the scenes that you don’t know about. Doing it by hand not only allows you to target more chips, but you can speed up a lot of routines, and save battery power.
So I thought I’d build one myself. The fanciest way would have been something to talk USB, which could have made all sorts of changes to the camera settings per-shot, but I went with the ultra-simple “bulb” interface. All you need to do is close the circuit between two contacts, present on an unscrupulously proprietary interface on the camera. A simple switch would work. So would a relay, if you want to automate it. But a relay uses quite a bit of current to operate, and one useful property any intervalometer can have is to run for a long time on a small battery.
So I elected to use an opto-isolator. These devices are totally electrically unconnected to the power they are switching, as they work by the input current switching on an LED inside an IC-style package, which shines onto a light-sensitive transistor, which switches on, allowing the current to flow through it. There is nothing other than light going between the two sides of the opto-isolator, and only in one direction at that!
I experimented to find what the shortest was that I could close the circuit and have the camera reliably take a photo, and wrote a short program to activate the pin that would turn on the opto-isolator for that time, and then sleep until time to fire again. I added an LED to visualise what was going on.
I designed the PCB in, erm, PCB.
This was printed out with a mono laser printer, onto the very glossy, flimsy inserts you get in newspapers, advertising, usually, cash for gold, or takeaways that sell EVERY FOOD EVER. Place this onto some clean, copper PCB board, and apply a hot iron. The kind you do shirts with, not a soldering iron. This melts the plastic toner binder, and it should preferentially stick to the copper. You then wash and very gently clean off the paper, and you’re left with the black toner where you want the copper to remain.
This goes into the etchant, which was hot ferric chloride in solution. Once this is complete, you can, again, carefully, clean off the toner, to re-expose the copper.
Next, drill the holes for the component pins. Highly recommend not doing this by hand, PCB drills are super brittle and will easily snap off. Then, solder the components in place. As a temporary housing, I think I used a box that some cufflinks came in.
The microcontroller was programmed with the following code:
[code lang=”c”]/*
* intervalometer.c
*
* Created: 22/07/2011 22:12:03
* Author: Phil Bambridge <phil@swedeheart.net>
*/
#include <avr/io.h>
#include <avr/interrupt.h>
#define FALSE 0
#define TRUE 1
unsigned volatile int systime = 0;
ISR(TIMER0_COMPA_vect) {
// Given our setup, this being called means a millisecond has elapsed
systime++;
}
void delay (int mssecs) {
int timenow = systime;
while(systime – timenow < mssecs) {
MCUCR |= 1 << 5; // Disable sleep guard
asm("sleep"); // We’ll wake up when the timer counts a millisecond and throws an interrupt
MCUCR &= ~(0|1<<5); // Enable sleep guard again
}
}
int main(void)
{
SREG |= 1 << 7; // Enable global interrupts flag
ACSR |= 1 << 7; // Turns off the unneeded analog comparator
PRR |= 1 << 0 | 1 << 1 | 1 << 3; // Turn off the ADC, the Serial and Timer 1 respectively
// Set up timer for a 1KHz interrupt- for 8MHz operation, that’s a 1/64 prescaler, and a count of 124.
// For 1 MHz (default) operation we’d go with a 1/8 precaler.
OCR0A = 124; // Count to 124
TCCR0A |= 1 << 1; // Clear timer when we hit that count set above
TIMSK |= 1 << 4; // Enable the interrupt
TCCR0B = 0x2; // 1/8 prescaler, also enables the timer-counter.
// Set all three output pins low, then set to output.
PORTB = 0;
DDRB = 0 | 1 << DDB0 | 1 << DDB2 | 1 << DDB3;
while(1) // Run forever
{
// Put shutter and indicator high
PORTB |= (1 << PORTB0 | 1 << PORTB3);
delay(75);
// Put shutter and indicator low
PORTB &= ~(0 | 1 << PORTB0 | 1 << PORTB3);
delay(2000);
}
}[/code]
By way of comparison, if I’d done this with an Arduino, the code would look something like this as a close approximation- it’s basically the blink sketch:
[code lang=”c”]#define OPTO 12
// the setup function runs once when you press reset or power the board
void setup() {
// initialize digital pin 12 as an output.
pinMode(OPTO, OUTPUT);
}
// the loop function runs over and over again forever
void loop() {
digitalWrite(OPTO, HIGH); // turn on the opto-isolator (open camera shutter)
delay(75); // wait for 75ms
digitalWrite(OPTO, LOW); // turn off the opto-isolator (close camera shutter)
delay(2000); // wait for a 2 seconds
}[/code]
As you can see, it’s a lot fewer lines of code. The Arduino preprocessor builds the real code after you hit the compile button (if/when you know C, you realise that what you see in the Arduino IDE can’t be the whole story), we can find the actual code in a temp file. In this case it was 8036 lines, 90% of which were comments. Not a bad read for a lazy Sunday afternoon. Point is, it’s doing lots of things behind the scenes that you don’t know about. Doing it by hand not only allows you to target more chips, but you can speed up a lot of routines, and save battery power.
Categories
My favourite regexp
Do you have a favourite regular expression? That might be a tricky question for some- like the benighted masses who haven’t yet heard the gospel of regular expressions. Or maybe you have so many dear to your heart, a real Sophie’s Choice? For me, it is easy, the first non-trivial one I wrote, for a task management system called TOM. Take a look and see if you can sell what it does- to help you out (?) I have left it in the context of the line of Perl it came from.
[code lang=”perl” light=”true”]$string =~ s/(?=.{79,})((.{0,77}[\-,;:\/!?.\ \t])|(.{78}))/$1\r\n/g;[/code]
Categories
Metametadata for steganography
So.
You’d like to slip some data past some prying eyes. For whatever reason, overtly encrypting the data isn’t possible. It’s not just that the data can’t be seen, but you can’t have anyone be aware data is even moving anywhere.
It’s not unheard of to use metadata for smuggling data. That way, the ostensible file- a Word document, a JPEG file looks innocent- it can even be opened up and read -but if you know where to look, there’s the secret data.
What’s data, in contrast to metadata? That’s a matter of intention. It’s the surface purpose of the file format.
But not all metadata is created equal.
The data itself can be crafted or written to convey a second meaning. For example, subtext in fiction, or taking a photo of 6 sunflowers.
You can use patterns- spaces or number of syllables in text, or colours of pixels in images.
You can encode the data in such a way that it includes extra meaning- using homoglyphs, or diacritic combining characters.
But we’re talking about metadata, I thought?
All metadata has a purpose, and is going to affect the way the file is read or displayed. Different applications may expose that metadata more visibly than others. So we have to be careful not to store our payload in what turns out to be plain sight. An example being the document title, which is often going to be displayed in the application title bar. So it might look a little suspicious in the document about cakes says “The X-94 Prototype uses Jumbillium alloy”.
One option can be to find formats where we can define our own metadata, or look for the equivalent of junk DNA.
What’s junk DNA?
Noncoding DNA or Junk DNA is DNA that doesn’t do anything (sidenote- much of it probably does do things after all). It just bulks out the genome to no effect (again, probably not true). But it sits right in there with the other DNA. Where most DNA shows the body how to make a protein, ncDNA does not.
Many file formats will have an end-of-file token, and ignore anything after that. They may have a payload length in the header, and again, everything after that is ignored.
Error correction codes have been misused in the past on audio CDs, but we could use the ECC bits to store our data- quite a percentage of the audio CD is that.
But what about metametadata?
Metametadata is metadata about metadata. It’s casting metadata as the main role, then thinking about what we can say about it.
You’d like to slip some data past some prying eyes. For whatever reason, overtly encrypting the data isn’t possible. It’s not just that the data can’t be seen, but you can’t have anyone be aware data is even moving anywhere.
It’s not unheard of to use metadata for smuggling data. That way, the ostensible file- a Word document, a JPEG file looks innocent- it can even be opened up and read -but if you know where to look, there’s the secret data.
What’s data, in contrast to metadata? That’s a matter of intention. It’s the surface purpose of the file format.
But not all metadata is created equal.
The data itself can be crafted or written to convey a second meaning. For example, subtext in fiction, or taking a photo of 6 sunflowers.
You can use patterns- spaces or number of syllables in text, or colours of pixels in images.
You can encode the data in such a way that it includes extra meaning- using homoglyphs, or diacritic combining characters.
But we’re talking about metadata, I thought?
All metadata has a purpose, and is going to affect the way the file is read or displayed. Different applications may expose that metadata more visibly than others. So we have to be careful not to store our payload in what turns out to be plain sight. An example being the document title, which is often going to be displayed in the application title bar. So it might look a little suspicious in the document about cakes says “The X-94 Prototype uses Jumbillium alloy”.
One option can be to find formats where we can define our own metadata, or look for the equivalent of junk DNA.
What’s junk DNA?
Noncoding DNA or Junk DNA is DNA that doesn’t do anything (sidenote- much of it probably does do things after all). It just bulks out the genome to no effect (again, probably not true). But it sits right in there with the other DNA. Where most DNA shows the body how to make a protein, ncDNA does not.
Many file formats will have an end-of-file token, and ignore anything after that. They may have a payload length in the header, and again, everything after that is ignored.
Error correction codes have been misused in the past on audio CDs, but we could use the ECC bits to store our data- quite a percentage of the audio CD is that.
But what about metametadata?
Metametadata is metadata about metadata. It’s casting metadata as the main role, then thinking about what we can say about it.
- GPS co-ordinates (which would be the metadata) of a location which can be looked up on a map, with a name, say, London, encoding for the letter L, or number 6. Or 50, I suppose.
- Or GPS co-ordinates where the least significant digits carry the hidden data.
- In databases, using wide, non-sequential ID columns to encode our data. The GUID type in Microsoft Access and SQL Server is 16 bytes wide! Feed our data through a 128 bit block cipher, and we’ll have pretty random-looking data, so given unique input plaintext, we are going to get unique cyphertext out. Prepending a sequence number to our text might be needed if we know we’ll be wanting to send repeating messages, but that eats into our budget. We can expose the GUIDs in some interface, perhaps in what would look like debug messages/comments in some HTML front end.
- TTL fields in IP headers. Now. These won’t survive end-to-end unaltered. But, over time, we can establish a baseline number of hops, interleaved with TTLs to carry our covert meaning. Sure, it’s not a lot of data, but with the right application, we can send a lot of packets in any session, or spread out over time- perhaps looking at other data leaving that network node or time of day to ensure it doesn’t look out of place. With the baseline TTL known (have to recheck periodically in case of path changes), we can monitor these transmissions from any node in the path.
- If we’re playing around at the packet level, we can also bury stuff in the sequence number of TCP headers, although this is going to require more co-operation at the other end, a network stack that can ignore sequence numbers on a certain port, perhaps, and simply funnels the sequence numbers to a decoding utility.
- Both of the above techniques could also be employed as a network “knock”, the idea of refusing connections to a port until a certain series of events happens, like a secret knock.
- Many file formats have a lot, or total flexibility with regards to the ordering of the metadata, so we can use that to encode data.
Problems with embedding data in metadata is that if it gets too big, the filesize is going to look incongruous with the overt data. It would be worth looking at a sample of JPEGs, etc, to determine the average, and have a think about what heuristics might be engaged to detect meta (or metameta) data payloads. We might scrub the EXIF clean of a JPEG, or at least flag it up as likely to be suspect, not just size, but use of unusual, or rarely used EXIF tags.