DONE 15aj/ When roll-forward finds a new segment, make sure youth number is
updated.
-15ak/ Load orphan file during roll-forward and make every block an
+DONE 15ak/ Load orphan file during roll-forward and make every block an
orphan.
-15al/ set filesystem update_time somewhere.
+DONE 15al/ set filesystem update_time somewhere.
-15am/ filesystem 'name' needs to be handled uniformly.
+DONE 15am/ filesystem 'name' needs to be handled uniformly.
DONE 15an/ can we be sure 'b' will be non-null in delete_inode?
-15ao/ determine what locking is needed to walk the children list
- in lafs_inode_handle_orphans. Probably the address_space private lock.
+DONE 15ao/ determine what locking is needed to walk the children list
+ in lafs_inode_handle_orphan. Probably the address_space private lock.
15ap/ Make sure write_inode has been cleaned up. See if this applies to
rollforward of a symlink (see FIXME)
-15aq/ change inode map to be little-endian, not host-endian
+DONE 15aq/ change inode map to be little-endian, not host-endian
-15ar/ understand what to do about errors in lafs_truncate
+DONE 15ar/ understand what to do about errors in lafs_truncate
15as/ handle errors from lafs_write_super ???
-15at/ More wait_queues to wait for different blocks.
- An array which we hash in to ??
+DONE 15at/ More wait_queues to wait for different blocks.
+ just use wait_on_bit / wake_bit
-15au/ How should iocheck_block set the page error?
- and block_loaded
+DONE 15au/ How should iocheck_block set the page error?
+ and block_loaded <- this gets it right.
15av/ ditto for write errors?
-15aw/ when lafs_incorporate makes a new block where the
+DONE 15aw/ when lafs_incorporate makes a new block where the
old is Realloc, the new should be Realloc too.
-15ax/ Think about what happens when we relocate a block
+15aw2 / When a block is a snapshot block it can never be dirty
+ so we only need credits for realloc...
+
+DONE 15ax/ Think about what happens when we relocate a block
in the orphan list (lafs_orphan_release), particularly
if the block isn't actually loaded.
+ FIXME still need to make sure errors will loading the orphan
+ file are handled correctly - I guess we mark all bad orphans as
+ type==0 and when we find those during release, reduce the size
+ of the orphan file.
-15ay/ Wonder if there is any way for orphan_run to get a wakeup
+DONE 15ay/ Wonder if there is any way for run_orphans to get a wakeup
when an inode or dir mutex is released.
+ No, there isn't.
-15az/ Sanity check all values in cluster head during roll-forward
+DONE 15az/ Sanity check all values in cluster head during roll-forward
i.e. in roll_valid. If the head isn't complete, we can still
use this to commit some previous checkpoints.
-15ba/ roll forward should not BUG on bad data like inodefile in
+DONE 15ba/ roll forward should not BUG on bad data like inodefile in
non-primary filesystem.
-15bb/ Do I need to sync something before copying an update over part
+DONE 15bb/ Do I need to sync something before copying an update over part
of an inode, then reloading the inode.
-15bc/ Handle DescHole in roll forward.
+DONE 15bc/ Handle DescHole in roll forward.
-15bd/ Call lafs_add_block_address from writeback rather than iolock
+DONE 15bd/ Call lafs_add_block_address from writeback rather than iolock
in roll forward, just for consistency.
-15be/ Confirm various files loaded at mount time (segusage, orphan ...)
+DONE 15be/ Confirm various files loaded at mount time (segusage, orphan ...)
are actually the correct type.
-15bf/ Avoid quadratics in lafs_seg_put_all - nothing else should be doing
+DONE 15bf/ Avoid quadratics in lafs_seg_put_all - nothing else should be doing
a lookup - or at least we can test for that.
lafs_seg_apply_all has similar problems and needs a good solution.
-15bg/ lafs_seg_ref_block is worried about losing implicit ref on parent
+DONE 15bg/ lafs_seg_ref_block is worried about losing implicit ref on parent
if parent splits. See what to do about that.
-15bh/ after roll-forward, check that free_blocks hasn't gone negative.
+DONE 15bh/ after roll-forward, check that free_blocks hasn't gone negative.
or handle if it has.
DONE 15bi/ Set EmergencyClean a bit later - need at least one checkpoint first.
to twostage.
-15bj/ Make sure .last link in segtracker is kepts uptodate, particularly in
+DONE 15bj/ Make sure .last link in segtracker is kept uptodate, particularly in
segdelete.
-15bk/ make sure get_cleanable doesn't lose a race before calling add_clean
+DONE 15bk/ make sure get_cleanable doesn't lose a race before calling add_clean
-15bl/ better checks for 'valid state block address' in valid_devblock
+DONE 15bl/ better checks for 'valid state block address' in valid_devblock
include that segment_count is credible
also in valid_stateblock
15bn/ How does refcounting of 'struct fs' work with multiple filesets?
-15bo/ use put_super to drop last refer to superblocks
+DONE 15bo/ use put_super to drop last refer to superblocks
-15bp/ review all superblocks - maybe use more anon??
+DONE 15bp/ review all superblocks - maybe use more anon??
15bq/ check readonly status in lafs_get_sb
-15br/ sync_fs should probably wait for something if 'wait'.
+DONE 15br/ sync_fs should probably wait for something if 'wait'.
+
+DONE 15bs/ set f_fsid properly in lafs_statfs
-15bs/ set f_fsid properly in lafs_statfs
+DONE - use new write_begin / write_end
- - use new write_begin / write_end
- - review how we ensure that credit remain with block.
+15bt/ - review how we ensure that credit remain with block.
15ca/ When pin inode data block, pin it as well as index block I think
It is still kept of the leaf list until the index block is done with
I think.
15cb/ Layout issues:
- - subset filesys still needs a parent pointer
- - cluster head needs mtime/ctime to log these.
+ DONE - subset filesys still needs a parent pointer
+ DONE - cluster head needs mtime/ctime to log these.
- need better tracking of which devices are in this array??
Need to be able to have read-only devices that are shared
- amove arrays.
- - need multiple parallel write-clusters to allow parallel writes.
+ among arrays.
+ DONE - need multiple parallel write-clusters to allow parallel writes.
- record tuning in state block:
- max_segs
- - use crc or something, not toy checksum (e.g. cluster - state already has)
- - flags for inconsistencies found, at layout/fileset/file levels(?)
+ DONE - use crc or something, not toy checksum (e.g. cluster - state already has)
+ - flags for inconsistencies found, at layout/fileset/file levels(?) (see 60)
- policies of whether old or new data is allowed on each device
- policies of how much duplication of metadata is required
- - inode map - not host-endian
+ DONE - inode map - not host-endian
+ DONE - segments > 16bit:
+ segusage file - what about youth?
+ cluster_head Clength
15cc/ free any stray B_ASync block found in destroy_inode
-15cd/ Some code assume a cluster header does not exceed 1 page.
- Is this safe? Is in true? Is it enforced?
+15cd/ Some code assumes a cluster header does not exceed 1 page.
+ Is this safe? Is in true? Is it enforced?p
+ roll-forward now handles large cluster_head.
+ Need cleaner to handle it, and need to possibly write large
+ cluster head when making new clusters.
15ce/ classify BUGs as
- internal logic errors
- unusual conditions I want a warning of
- data corruption errors
-15cf/ lafs_iget_fs need to sometimes to in-kernel mounts for subset filesystems
+DONE 15cf/ lafs_iget_fs need to sometimes to in-kernel mounts for subset filesystems
This is needed for the cleaner - the cleaner needs to hold a ref somehow.
15cg/ lafs_sync_inode is weird - why the lafs_checkpoint_start and update_cluster
15ch/ Review values of youth and checkpoint_youth and think about off-by-one
issues.
+15da/ Replace directory updates!!!!!
+
+15db/ Decide how version string will be used.
+
+15dc/ resolve table_size - it should be stored in the segusage file and validated
+ based on device geometry.
+
+15ea/ rollforward should recognise VerifyDevNext{,2} to allow next
+ cluster on same device to verify previous.
+
+15eb/ When multiple devices and lots to do and plenty of free space,
+ allow multiple segments, one per device, to be open at once,
+ and possibly be writing multiple clusters at once using
+ VerifyDevNext2
+
+15ec/ Implement i_version tracking. This should be a 64bit numbers
+ that appears to change every time the file changes. We only
+ need a new number when someone looks at the value with
+ getattr.
+ We could simply use mtime with the sub-millisecond part being
+ a counter of times that getattr sees a change in the same
+ millisecond.
+ However as mtime can go backwards we might get i_version going
+ backwards, which is awkward. I wonder if I care.
+ Otherwise, leave for an inode extention later.
+
16/ Update locking.doc
17/ cluster_flush calls lafs_cluster_allocate calls lafs_add_block_address
- reading critical metadata
...
-23/ Rebase on 2.6.latest
+23/ Rebase on 2.6.latest. Done for .38
24/ load/dirty block0 before dirtying any other block in depth=0 file,
else we might lose block0
31/ Implement non-logged files
-32/ Store access time in non-logged file
+DONE 32a/ Store access time in a file
+32b/ Make it a non-logged file
+32c/ Avoid writing out dirty atime file blocks when not necessary.
+ i.e. keep the page clean and active, and trigger 'write'
+ on release_page.
33/ Support quota : group / user / tree
- how to destroy
36/ review roll-forward
- make sure files with nlink == 0 are handled well
- sanity check before trusting clusters
+
+DONE 36a/ make sure files with nlink == 0 are handled well
+DONE 36b/ sanity check before trusting clusters
+DONE 36c/ handle miniblocks which create new inodes.
+DONE 36d/ Handle DescHole in roll_block
+DONE 36e/ When dirtying a block in roll_block, maybe use writeback rather
+ than just iolock, for consistency...
+DONE 36f/ What to do if table becomes full when add_block_address in
+ roll_block ??
+DONE 36g/ Write roll_mini for directories.
+DONE 36h/ In roll_one, use the cluster counting code to find block number and
+ make sure we don't exceed the segment.
+DONE 36i/ add more general error checking to lafs_mount -
+ lafs_iget orphans and segsum. Check type is correct.
+ errors from lafs_count_orphans or lafs_add_orphans.
+ alloc_page failure for chead - maybe allocate something bigger??
37/ Configure index block hash_table at run time base on mem size??
44/ Examine created filesystem and make sure everything looks good.
-45/ mkfs.lafs
+DONE 45/ mkfs.lafs
46/ fsck.lafs
56/ Review roll-forward completely.
57/ learn about FS_HAS_SUBTYPE and document it.
+ This is for fuse in particular so users can know the real type
58/ Consider embedding symlinks and device files in directory.
Need owner/group/perm for device file, but not for symlink.
Can we create unique inode numbers?
hard links for dev-files would be problematic.
+ What do we gain? Maybe something for short symlinks.
+ 40 seems a good length to get 70% of symlinks.
59/ Fix NeedFlush handling so we don't drop-then-retake
- a mutex and that isn't sensible.
+ a mutex as that isn't sensible.
60/ Introduce some fs state recording that fsck is needed and possibly
identifying what sort of fsck.
realloc or dirty rather than lafs_allocated_block doing it.?
See also 15ad below.
+66/ Delay writeout of directory updates until an fsync. If a checkpoint happens
+ first, discard the updates (and fsync waits for checkpoint to complete).
+ If a cross-directory rename happens care is needed: either flush updates
+ first or ensure that a flush does happen before the cross-directory
+ update is flushed.
+ Note that if the target of a rename is a directory, it must also be fully
+ flushed before the rename can proceed.
+
26June2010
Investigating 5a
The writeout can be much later, but logging the mtime is fairly
boring ... we could log mtime in the group head, which might be cheap
enough. How much precision is needed, and against what base?
- probsably mtime of last checkpoint from superblock. That should
+ probably mtime of last checkpoint from superblock. That should
be not more than 2048 seconds ago, so 16 bits gets is 30msec...
14Aug2010
So as we find a new segment we want to do much like was lafs_free_get does youth_update.
But the data block - isn't that youthblk? When it that set?
segsum_find sets if it ssnum == 0
+
+19sep2010
+ 15ak - run the orphan file at mount time.
+ After roll-forward when we have a working filesystem, we need to read the orphan file, load each block
+ mentioned, and register each as an orphan.
+ This involves:
+ - setting the orphan_slot
+ - setting B_Orphan
+ - lafs_add_orphan
+ Just like at the start of orphan_commit
+ We also need to initialise nextfree and possibly 'reserved'.
+ But: can orphans be created during roll-forward? They certainly can. We currently hide that in a re-use of
+ the orphan list.. But directory updates are possible too, and not handled.
+
+ I guess we should examine the file as soon as root is loaded as before roll-forward as roll-forward cannot
+ change the orphan file. Then after roll-forward, we read the original part of the file and set up
+ any orphans that aren't yet.
+ So we want to read once to get the size. Then read again to process content up to that size.
+
+ 15am - filesystem name.
+ This is only used for identifying snapshots
+
+01oct2010
+ - mkfs is done to an initial version of lafs-utils. !!!
+
+ So: 15am - filesystem name - used to identify snapshots
+ So the name is pointless in subordinate filesets. So I could just shrink
+ the metadata. The primary metadata needs to be big enough to get a name
+ easily though.
+
+ 15aw..
+ When cleaning we have a separate credit bit 'B_Realloc' from 'B_Dirty'.
+ But we have the same B_UnincCredit bit for both. Is that safe?
+ Processing the cleaner could absorb the UnincCredit while the blocks is
+ reserved but not dirty. Then when it gets dirtied, there may be not
+ enough credits to split.
+ We set Dirty from Credit, and use ICredit for UnincCredit.
+ But when only Realloc (not dirty) we don't use those bits. We allocate
+ fresh credits or set Dirty if that fails.
+
+03Oct2010
+ Need lafs_iget_fs to work on other filesystems. And other snapshots?
+ We use it:
+ in cleaner when parsing cluster head
+ in orphan handler when loading orphan file or when rearranging it.
+ in roll forward
+
+ Each of these might need to kern-mount the fs - so we need to hold the ref
+ somewhere.
+ Cleaner also needs to explore snapshots.
+
+ Don't want kern_mount - that is too heavy weight and includes a vfsmnt.
+ Just split up lafs_get_subset and use sget etc. so we get an 'sb' that we need
+ to hold.
+ Similarly for snapshots. Cleaner needs to consider all snapshots, so they
+ all need to be mounted.
+
+ So snapshot 'sb's are referenced by cleaner, and de-reffed when cleaner stops.
+ Subset 'sb's can be attached to the parent inode and then only dropped when
+ the inode goes... only sb currently references inode.
+ So maybe the first ref to an sb doesn't ref the inode but others do - is that
+ possible? No, as we don't see them being dropped.
+ Every inode in the subset could ref the filesys inode. That would keep it active
+ the right amount of time, but release/destroy could still be racy.
+
+ I guess cleaner/orphan/roll need to explicitly ref the fs.
+ cleaner already refs inode when B_Cleaning, so hold fs too.
+ B_Orphan seems to own and inode ref too.
+
+ So:
+ lafs_iget_fs gets a ref on the inode and the sb.
+ need lafs_iput_fs to drop both references
+ B_Cleaning, B_Orphan, I_Pinned and I_Trunc all hold this double ref.
+
+ cleaner holds refs on all snapshots
+
+ FIXME I probably need to hold inode/fs for B_Async too.
+ No. Async only refs the block, not the inode or fs.
+ Something else would normally ref the inode - e.g. cleaner.
+ When the inode is free, the page invalidation will notice the
+ B_Async flag and release it.
+
+ So that is all done now, except I don't hold refs on snapshots in the cleaner
+ yet.
+
+11oct2010
+ DescHole
+ - When is this used? directory etc don't need it.
+ - a regular file might, but there is no API to punch
+ a hole.... yet I guess.
+ - So we just want to allocate these blocks to 0.
+
+15oct2010 - happy birthday Daniel...
+ Looking at 36:
+ a/ files with nlink==0;
+ If we happen to find them, we hold a reference until all roll-forward
+ is done, incase a name is found - it is important not to start deletion
+ early.
+
+18oct2010
+ 36g - write roll_mini for directories.
+ We get a name, an inode number, and one of:
+ LINK UNLINK REN_SOURCE REN_NEW_TARGET REN_OLD_TARGET
+
+ The REN_SOURCE is linked with a REN_*_TARGET which could be in a
+ different directory, so we need to stash the SOURCE until the TARGET
+ arrives.
+ We simply impose the implied change on the directory and update the
+ link count in the target inode.
+ So:
+ load the inode
+ possibly record REN_SOURCE for later
+
+ calls prepare/pin/commit as appropriate.
+ Put the inode on orphan list if appropriate - needs care
+ as we retarget orphan list.
+ update inode link count.
+
+ (28Feb2011)
+ Just a refresh on the purpose of these updates.
+ 1/ They allow us to fsync a directory without performing a full checkpoint.
+ As directory blocks are not processed in roll-forward we need the update
+ for data to be safe. As fsync of directories are rare in some common
+ situations we could avoid actually writing these. Simply queue them
+ internally and discard them on a checkpoint. If an fsync comes before the
+ checkpoint, only then do we write them out. If there are any cross-directory
+ renames then the preceeding updates in both directories need to be flushed
+ before the cross-directory rename. It might be easier to always flush on
+ a cross-directory rename.
+ 2/ They ensure consistency of inode link-count wrt to names in the filesystem,
+ but as link count is only updated by these (or a checkpoint) there is no
+ problem with delaying.
+
+ So: when replaying these we must update the directory content and the inode
+ link count.
+ It is OK to delay the write-out of these until an fsync, and not bother
+ if a checkpoint happens.
+ So add that to th TODO list - item 66.
+
+28feb2010
+ - roll forward directory updates ... I wonder if I got it right :-)(untested).
+
+
+ I don't seem to have easy-access notes about the various meaning of
+ 'width' and 'stride'
+
+ width: The number of independent devices across which the (virtual) device
+ is placed. The normal goal is to write 'width' blocks on every single write.
+ On a RAID4/5/6 this will avoid the need to pre-read for parity calculations,
+ and it will keep all devices equally busy with writes.
+ The 'width' blocks probably aren't consecutive.
+
+ There are two different layouts - one with width*stride <= segment_size
+ and one with width*stride > segment_size.
+
+ width*stride <= segment_size
+ This is a traditional striped layout like RAID0/4/5/6.
+ The 'stride' is the chunk size, so 'width*stride' is the stripe size,
+ and segment_size must be a multiple of this.
+ In this case all addresses in a single segment are contigious. We don't
+ necessarily write them in order if we want to write less than one stripe.
+ segment_offset will normally be a multiple of width*stride though this isn't
+ enforced as one could have a partition with an non-aligned start.
+
+ width*stride > segment_size
+ This implies a catentated layout. If parity-redundancy is in use when the
+ blocks which combine to form a stripe are 'stride' blocks apart.
+ The benefit of this layout is that an extra drive can be added by simply
+ zeroing it and joining it to the array - no re-stripe needed.
+ This will make all stripes slightly larger so at first the space will not
+ be available. As cleaning happens the space will gradually become
+ available. This still requires restriping, but unlike a normal
+ raid5 restripe, the space becomes available in small amounts immediately,
+ when there is no demand for more space, the re-striping (cleaning) can happen
+ at a very low priority with no cost.
+
+ In this case the blocks in a segment are not contiguous.
+ 'segment_size/width' are, then there is a large gap (in virtual address
+ space) to the next chunk.
+
+ The segment_offset is an amount of space which is free at the start of
+ each device. 0..segment_offset and stride..stride+segment_offset etc
+ do not contain data and can be used for metadata.
+
+ When width > 1 it makes sense to replicate each state block across
+ every device - as we want to write the whole stripe anyway.
+ For now we only write and read the first two copies at the beginning, and
+ the last two at the end...
+
+ Question: what do we want to do about metadata on flash devices? We really
+ don't want a small number of locations to store the metadata, but a large
+ number that we search through - possibly a binary search.
+ These could be all at start/end or scattered throughout the device.
+ The later would make it impossible to find efficiently - there is no way to
+ create useful linkage without writing something else at start of end.
+ As many devices optimise for random writes where the FAT table would be,
+ it make sense to just put the metadata there and not at the end.
+ We should allow one 'page' for each metadatum, which probably meanss
+ 32K.
+ So we should allow all state blocks to be near the start.
+
+01mar2011 - Autumn arrives.
+
+ Time to add handling of 'atime' and non-logged files.
+
+ The idea is to have a separate file for storing only 'atime'
+ This is separate from the inode file because the volatility of the data
+ is very different and one of the principles of log-structured-fs is that
+ differently volatile data should be kept separate.
+
+ This does mean that an inode lookup requires getting data from two files,
+ but it is hopped that the 'atime' file will mostly be in cache as each
+ block contains the atime for lots of different inodes.
+
+ The atime file contains 2 bytes for each inode, so with a block size of 4K,
+ each block would hold info for 2048 inodes. 1 million inodes would require
+ 2 megabytes.
+
+ The 16bits are treated as a positive floating point number which
+ gets added to the atime stored in the inode. The lower 5 bits are
+ the exponent, the remaining 11 bits are mantissa. Though there is a
+ little complexity in interpreting the exponent.
+ If the exponent is 0, the mantissa is used as milliseconds -
+ so shift left 5 and multiply by 1000000 for nanoseconds.
+ The smallest change that can be recorded in 1 millisecond.
+ and values up to (2^11-1) milliseconds - or 2seconds can be stored.
+ If the exponent is 1 to 10, the mantissa has a '1' appended as a
+ new msb, and is shifted by the exponent-1 and then treated as milliseconds.
+ This ranges up to 2^(12+9) milliseconds or 30 minutes, where
+ the granularity will be 2^9 millisecs or 0.5 seconds
+
+ For exponents from 11 up to 31 we add the 1 msb and treat
+ the number as seconds after shifting (e-11). So at e==31,
+ we shift a number that is
+ up to 4095 by 20 to get nearly 2^32 seconds or 136 years.
+ At this point the granularity is 2^20 seconds or 12 days.
+
+
+ So overall we can update the atime for 136 years without needing to
+ update the inode, and can record differences of 1msec for the first
+ couple of seconds, then gradually less granularity until we are
+ down to one second an hour after the last change, and 4 hours a
+ year later.
+
+ To convert a number of seconds to this format:
+
+ If >= 2048 seconds, we shift down until less than 4096 seconds
+ counting the shift. We add 11 to that number to form exponent,
+ and shift the resulting mantissa up 5, or with exponent, and mask
+ out bit 16.
+
+ Otherwise we convert to milliseconds (divide nanno by 1000000 and
+ multiply seconds by 1000, and add). Then if < 2048, we shift up by
+ 5 leaving a zero exponent and use that.
+
+ Otherwise we shift down until < 4096 counting shifts, add 1 to the
+ shift to form an exponent, and combine with mantissa as above.
+
+ So that is the format - how do we implement it?
+
+ We don't want to expose to user-space numbers that we cannot store.
+ So any 'utimes' call updates that the inode directly can clear the
+ value in the atime file. Only updates due to accesses go to the atimes
+ file.
+ We define a 'getattr' function which looks at the atime stored in
+ the vfs inode and if it has changed we need to deal with it.
+ - if the inode is still dirty we simply update the lafs inode
+ and use the number as-is, clearing the atimes entry
+ - else we subtract the stored atime from the new atime. If this
+ is negative or exceeds 136 years we mark the inode dirty and
+ store it there. It we cannot mark the inode dirty for some
+ reason we just store all 1s in the atime file.
+
+ The same operation is needed when dirty_inode is called to make
+ sure atime updates get saved even when no getattr is called.
+
+ As we always need to be able to update the atime file, it needs to
+ be permanently pinned whenever an inode is read in. For
+ non-logged files this should be cheap but we must do it anyway as
+ the file might not be non-logged.
+ So we need to keep a permanent reference to each block while the
+ inode is loaded. That can keep it pinned.
+
+
+ We don't want updates to the atime file to be flushed in any great
+ hurry, especially if it is a logged file. We would be quite happy
+ to only write at 'unmount' and probably 'sync'.
+ So we want to stop the pages from appearing dirty in the page
+ cache (PAGECACHE_TAG_DIRTY), and the inode from appearing dirty
+ (I_DIRTY).
+ We can still keep them dirty in lafs metadata so if release_page
+ is called we can schedule a write out then.
+
+
+ So some steps:
+
+ 1/ load atime file at mount time - there is one for each
+ filesystem. It has inum of 3 and type of TypeAccesstime (6).
+ Also release it on unmount.
+
+ 2/ loading an inode must take a ref to the block in the atime file
+ if it exists. A new inode flag records if this has happened.
+ Unless mounted noatime, we pin the block and reserve space.
+
+ 3/ getattr and dirty_inode must resolve any issues with the
+ atime. So lafs_inode probably needs an extra field to be able
+ to check for changes
+
+
+
+ Hmm.. this is getting confusing...
+ When atime is changed the only way we find out is by ->dirty_inode
+ being called. But that is called when anything is changed.
+ Filtering out whether or not we need to update the inode itself
+ is awkward... maybe there is some context we can use.
+ ->dirty_inode is called by mark_inode_dirty which is called:
+ - by touch_atime, if something changed
+ - file_update_time - at which time we also update iversion
+ - setattr ... which has changed recently (2.3.37ish)
+ - page_symlink
+ - generic_file_direct_write - which increasing size of inode
+ - set_page_dirty_nobuffers
+
+ So either the inode is pinned, or it isn't.
+ If it isn't, then this *must* be an atime-only update.
+ If it is, then it could be anything, but in any case we update the
+ atime directly.
+ So: dirty_inode should try to get dblock and check if it is pinned.
+ If it is pinned, then update the atime immediately and the offset
+ in the atime file too.
+ If not, just update the offset
+
+
+03mar2011
+ ARGggg... checkpin is interfering with unmount - it keeps an
+ s_active count so unmount 'works' but doesn't release anything.
+
+ checkpin is needed is needed to ensure that inodes remain safe while
+ we are cleaning. Particularly, while the inode index block is
+ pinned, we keep the inode and fs referenced as well. I guess the
+ theory is that they won't stay pinned for long - but they do.
+ e.g. segusage blocks are permanently pinned.
+
+
+ We could have a rule about the prime filesystem always being mounted.
+ Then we don't need refcounts, but kill off the cleaner before
+ unmount... which we sort-of do..
+
+ All subordinate filesystems have references on the prime_sb so the
+ prime_sb must be the last one to go. When it goes it kills
+ everything off...
+ So we don't need checkpin to take a ref on the prime_sb.
+
+ There might be still an issue with files in subset filesystems
+ being permanently pinned so they stay around longer than they
+ should... need to check on that somehow.
+ The idea is that a quota file block is permanently pinned so it
+ will keep the fs pinned. That in turn will keep everything else
+ pinned... Worry about that when we implement quotas FIXME
+
+04mar2011
+ I really need to sort this out, and it isn't easy...
+ We really want to know when "all" filesystems have been unmounted
+ so the block device(s) can be released and the cleaner stopped.
+ But we don't have a count for that. We could if that was all
+ we counted - but that would mean that we only have a single
+ struct super_block for all filesystems.
+
+ So that is what I have to do. A single super_block for all parts
+ of the filesystem. I probably still need to allocated other
+ dev numbers stat->dev, but I don't need to use them internally.
+ Maybe I even allocate superblocks... Yes - we need to use
+ set_anon_super and kill_anon_super to allocate the numbers.
+ lafs_inode will need a pointer to the filesystem - we use that
+ instead of the sb.
+
+ -------
+
+ Testing...
+ bug at block.c:658. Block not B_Valid in lafs_dirty_iblock from
+ lafs_allocate_block from cluster_flush.
+ Block is 74/0: InoIdx block of a newly created file I think.
+ '74' was /f23, then /mnt/1/adir. We are creating file in that
+ dir.
+ This is a depth=0 InoIdx block - i.e. the data is in the
+ dblock, so there is no index info, so it kind-a makes sense for the
+ index block to not be Valid.
+ yes- commit d268a566605bf006cf33c confirms that.
+
+ So why are we trying to dirty it?..
+
+ Maybe:
+ We create a couple of directory entries, then flush and end up
+ with an in-line data block.
+ Then we add more, flush again and so try to dirty parent...
+ Where to we turn depth=0 inodes to depth=1??
+ - erase_dblock_locked - don't want that
+ - lafs_incorporate
+ So I guess the 'bug' is in error - it is OK to mark that invalid
+ block as dirty.
+
+04mar2011
+ So - back to the super_block reworking. We want only one
+ superblock.
+ So we use the TypeInodeFile inodes a bit more to hold the details
+ of different filesystems. We need to store a unique 'dev' number in
+ there use set_anon_super/kill_anon_super on a local 'struct
+ super_block' and copy s_dev in/out.
+
+ As we only have one sb, we can only have one fstype, so we cannot
+ use the fstype to choose what to do.
+ - if dev_name is a block device we try an normal mount
+ - if dev_name is a Inode file, we perform a subset mount
+ - if dev_name is a lafs dir and '-o snapshot=name', we mount that
+ snapshot
+ - if dev_name is a lafs dir in root with perm zero and
+ '-o subset=MAXSIZE', create a subset filesystem.
+
+ - lafs_iget needs an inode rather than a superblock
+ ditto for lafs_new_inode, lafs_inode_inuse, inode_map_free,
+ choose_free_inum, inode_map_new_prepare
+ - lafs_iput_fs,lafs_igrab_fs, ino_from_sb
+
+ - NFS filehandles need careful thought
+ They are 'per-super-block', not 'per-vfsmnt' which might be
+ better.
+ We could change that but.....
+ For non-snapshot files it is easy - just record two inodes, the
+ fs and the target.
+ For snapshots there is nothing that is really stable.
+ Maybe we could have different superblocks for snapshots.
+ The snapshot doesn't need the cleaner as it is read-only, though
+ the cleaner can need the snapshot...
+
+ So the cleaner might automagically mount a snapshot, but a
+ snapshot will never invoke the cleaner or any other thread stuff.
+
+ So I guess we want one superblock for the fs and one for each
+ snapshot.
+ The filehandle is then either inum+gen or inum+inum+gen where first
+ inum must be TypeInodeFile
+
+07mar2011
+ ... though I could just put a snapshot number and partial timestamp
+ in..
+
+
+08mar2011
+ This isn't a new to-do list, it is a list of the main features that are
+ still not implemented:
+ - full 2D layout
+ + at very least I don't pad with zeros yet
+ + if stripe size were multiple of 3*3*5*7*2^N, then changing
+ width might be managable.
+ e.g. stripe size: 40320 blocks.. But with megabyte chunksizes,
+ we really want 32bit segsizes and 322560 block segments.
+ - non-logged files - with interface to request access-time file
+ - quotas
+ - snapshots: particularly cleaning
+ - error handling
+ - metadata (inode/directory/etc) CRCs and duplication
+ - fsck / debugfs
+
+
+ What would fsck do?
+ - locate and validate device and state blocks.
+ - locate and validate checkpoint cluster.
+ - locate and validate filesystem root
+ - roll forward to collect segusage and quota blocks.
+ - load inode map, read inode file, validate each inode and make sure
+ map is correct.
+ - explore each file, following all indexing, count segusage for each
+ segment and make sure segusage file is consistent.
+ - check no block is allocated twice. This might require multiple passes,
+ each time we examine a different collection of segments.
+
+ - checking a file requires:
+ - checking inode is consistent
+ - checking index blocks are consistent with depth
+ - checking index/extent blocks are sorted with no overlaps
+ - checking block/iblock counts are correct.
+ - checking all cluster headers in the current segment to ensure they
+ look consistent and agree with file information. i.e. if cluster_header
+ identifies a block, the block must live there, or later in the segment.
+
+ - scan all directories looking for consistency of hash etc. Count links
+ for all inodes. This might need to be multi-pass too.
+ Could use a bitmap for single-link files, and table for others.
+
+ How to fix errors.
+ - First must find segments which are not in use according to segusage file
+ or according to block search.
+ If there are none, require a new device be provided.
+ - If anything looks incorrect, write corrected version to new segment
+ Then write out new segusage files
+
+ In some cases we might need to search all write-clusters for missing blocks??
+ That could take a very long time!
+
+
+ What do I really want to do about CRCs and hashes.
+ It might be nice to store a hash for each block in the index block.
+ But that wastes precious index-block space.
+ If I store a CRC together with address info in the block, then I could
+ be fairly sure it is the right block. So e.g. inodes store the inode number,
+ Index blocks could hold inode+depth+address.
+ Last 8 bytes of each block could be a 4byte CRC and a 4byte identity.
+ identiy is XOR of fsinum inum blocknum generation - or a CRC of these.
+
+ Actually, we don't need to store the identity info - we just need to
+ include it in the CRC. That either saves space, or allows more bits to
+ be used for the CRC, which is probably the best use of bits for detecting
+ errors.
+ Though it might be nice to store phys-addr in the CRC too, we cannot as
+
+21mar2011
+ My short-term todo list is:
+DONE - get 'lafs' to the stage where I can create an fs requiring roll-forward
+DONE - use 'lafs' to create images for testing, so I don't need 'fred.safe' any more.
+DONE - Make lots of 'layout' changes - see 15cb
+
+02may2011
+ - 'run' goes to completion, but segusage isn't updated in the final cluster
+ and the number left over from before looks wrong.
+DONE - 'ls -l' on a subset file gets confused.
+ - fs created by 'lafs' has wrong Blocks and Inodes counts
+ - we lose a ref to a segsum and sometimes put it too often.
+REFCNT 1 [ce0ffc48]0/182(2535)r0E:Valid,Claimed,PhysValid NP
+REFCNT 1 [ce055b9c]0/187(2535)r0E:Valid,Claimed,PhysValid NP
+REFCNT 1 [ce0445d8]0/182(2535)r0E:Valid,Claimed,PhysValid NP
+
+
+03may2011
+ Once I have these bugs sorted out I want to make some format changes.
+
+ DONE - fs_metadata need a 'parent' link
+ rename needs to be careful about what is updated!
+ so does roll_mini
+ lafs_get_parent needs some thought.
+
+ DONE - roll-forward should get exact mtime stamps, and ctime.
+ So each data block must have an exact timestamp
+ of when the change actually happened. Or the group_head
+ has a timestamp for the most recent update to the file
+ As we use nanosecond timestamps (pointless though they are)
+ we need 30 bits for the nanoseconds and at least 11 for the seconds.
+ So 48 bits (6 bytes) is plenty.
+ So include a 64bit timestamp in the cluster_head and 48bit
+ number to subtract in the group_head
+ But saving 2 bytes per file isn't really worth it, and we may
+ well lose it in padding. So just store a 64bit timestamp in
+ the group_head.
+
+ DONE - use CRC in place of all checksums - lafs_calc_cluster_csum
+
+ DONE - state block flags for inconsistencies found
+ If any inconsistency found, fsck is advised.
+ For some it may be imperative.
+ Things that can be wrong include:
+ - generic read error
+ - segusage negative
+ - index block incoherent
+ - dir block incoherent
+ - link count negative
+ - cluster header incoherent
+ -
+ 64 bits should be adequate and simple for this.
+ Any unknown bit requires a full fsck.
+
+ DONE - 32bit segment size
+ With 16bit at 4K blocks we are limited to 256Meg segments.
+ 64Meg with 1k blocks. This takes about 1 second to write on
+ a modern drive. On an array it will take even less time.
+ 24bits gives 16 to 64 gigabytes which is plenty.
+ However 24bits is awkward to access. a 1K block holds 341 1/3.
+ A 4K block holds 1365 1/3.
+ But this wastes less space than 256 or 1024 and so causes less IO.
+ But then we probably want to size segments to be very big.
+ A few thousand segments should be OK, which is tens of blocks.
+ I don't think the savings with 24bits are worth it, and I do
+ think v.big segments could be useful, so lets go with 32bit segments.
+
+ Youth is currently tuned to 16bits. Let's leave it there and
+ maybe waste some space.
+
+
+ - parallel new-data write clusters.
+ I think it is sufficient to include a second 'next_addr' in the
+ cluster_head - or maybe two. alt_next_addr[2].
+ When a thread wants to start a new stream of clusters it allocates
+ the segments then attaches to the next outgoing write cluster.
+ Once that is written everything in the new cluster is safe.
+ On a checkpoint every stream writes at least one checkpoint cluster
+ and these are linked together through alt_next_addr.
+ The 'next' cluster for each must be the checkpoint cluster and must
+ carry linkage but unlike with first-link, there is no need to wait
+ The data is already safe as long as the state block isn't updated
+ until every cluster_end block is written.
+ So really, one is enough. I had though 2 would enable quick fan-out
+ but there is no real need for that.
+
+ As 0 is a valid write-cluster address we use 'this_address' to signify
+ that there is no alt-next.
+
+ It is possible that a block of a file could be written to two
+ different streams at different points in time between two checkpoints.
+ We need to ensure that roll-forward gets these in the right order.
+ 'seq' can be the same in two different streams so we cannot use that.
+ timestamp could possibly be used, but as times can go backwards it
+ is not ideal.
+
+ NEW IDEA. Just use one stream of clusters. However it can
+ bounce from one device to another easily. So two different
+ threads can be building up two different write clusters at the
+ same time as long as they synchronise at some point to pass
+ addresses around. They also need some other Verify mode as
+ VerifyNext or VerifyNext2 will destroy any parallelism.
+ As the point of this is two write to multiple devices in
+ parallel, maybe VerifyDevNext{,2} meaning the next header on
+ the same device serves to verify this.
+
+ - policies.
+ This includes
+ maximum number of segments written between checkpoints
+ whether data can be cleaned to a particular device
+ whether a device can receive new data
+ whether metadata duplication is needed
+ whether an RO device from a different array is allowed.
+ Some of these are per-device policies. Some are per-array.
+
+ The 'RO Device' thing is special. I think I want an alt_uuid.
+ It works like this: You assemble the RO array when you
+ mount a new filesystem identifying the old as a component.
+ So that 'state' block on the new devices must identify the alt_uuid
+ and state seq number.
+
+ Do we want to record more info about which devices are in the
+ array? Currently we just record how many. If we find enough
+ with the right UUID/seq, they must be it.. what else would we
+ want?
+
+ For all the other policy statements it is probably simplest to
+ allow a set of simple strings. e.g. "noclean", "nonew",
+ "dup=2" "maxseg=5"
+ devblock currently uses 146 bytes, so room for 878
+ stateblock uses 112 plus some for snapshots, so much the same.
+ We currently don't use 'version' and have no concrete plans.
+ The vague idea is to allow lafs to *know* that it cannot mount
+ the array, so any incompatible feature gets set.
+ We could keep those in the policy sets. From that perspective
+ there are 3 types of things.
+ - if you don't understand, don't worry
+ - if you don't understand, don't try to write
+ - if you don't understand, you cannot even read.
+
+ That last is really best avoided. We have version info
+ elsewhere in the tree so that a new index style will simply
+ make that block unreadable.
+ So I think make the dev and state blocks a simple incrementing
+ version number which apply to that block, and have "don't
+ worry" and "don't write" policies distinguished by first
+ letter.
+ Capital is "If you don't understand, don't write"
+ Lower is "if you don't understand, don't worry".
+
+ These are space separated strings
+
+ - etc.
+
+ - what about i_version? Include in timestamp?
git / LaFS.git / blobdiff